Skip to content

use GCS mirror for Maven, fix benchmarks version, other small changes #166

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rapids-bot merged 1 commit into from
Jun 26, 2026
Merged

use GCS mirror for Maven, fix benchmarks version, other small changes #166

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .mvn/maven.config
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
-e
-B
-Daether.connector.basic.downstreamThreads=1
-Daether.transport.http.retryHandler.count=5
-Daether.transport.http.retryHandler.interval=10000
-Dmaven.wagon.http.retryHandler.count=5

@jameslamb jameslamb Jun 24, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See NVIDIA/cuvs#2253 (comment) for an explanation of these flags.

I believe comments are not allowed in this file because its contents are passed literally to mvn.

9 changes: 5 additions & 4 deletions .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION.
# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

@jakirkham jakirkham Jun 25, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the context behind this addition?

@jameslamb jameslamb Jun 26, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was added automatically by the verify-codeowners pre-commit hook.

This became the default in February (rapidsai/pre-commit-hooks#120) based on internal conversations about the preferred form.

It's showing up here now because:

  1. that change went into rapidsai/pre-commit-hooks v1.5.0 and this repo was using an older version until this PR
  2. the hook only changes copyright statements on files as they're touched, not all at once, and this PR touches that file

If you have questions about the history behind rapidsai/pre-commit-hooks#120 you can talk to @KyleFromNVIDIA . But this has been the standard across RAPIDS for a few months now.

@jakirkham jakirkham Jun 26, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this in other files then?

A few files here changed, but not all have this particular wording

@jameslamb jameslamb Jun 26, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know why this project doesn't include .xml files in its configuration for that hook. It's probably unintentional, I'd be happy to approve a PR adding them if you want to put one up.

# SPDX-License-Identifier: Apache-2.0

repos:
Expand All @@ -9,8 +9,9 @@ repos:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-symlinks
- id: check-xml
- repo: https://github.com/rapidsai/pre-commit-hooks
rev: v1.4.2
rev: v1.5.1
hooks:
- id: verify-copyright
name: verify-copyright
Expand All @@ -24,7 +25,7 @@ repos:
dependencies[.]yaml$|
^[.]pre-commit-config[.]yaml$
- repo: https://github.com/rapidsai/dependency-file-generator
rev: v1.20.2
rev: v1.21.0
hooks:
- id: rapids-dependency-file-generator
args: ["--clean", "--warn-all", "--strict"]
Expand All @@ -33,6 +34,6 @@ repos:
hooks:
- id: shellcheck
- repo: https://github.com/zizmorcore/zizmor-pre-commit
rev: v1.24.1
rev: v1.26.1
hooks:
- id: zizmor
3 changes: 3 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,19 @@ This library provides a new [KnnVectorFormat](https://lucene.apache.org/core/10_
## Building

### Prerequisites

- [CUDA 12.0+](https://developer.nvidia.com/cuda-toolkit-archive),
- [Maven 3.9.6+](https://maven.apache.org/download.cgi),
- [JDK 22](https://jdk.java.net/archive/)

```sh
mvn clean compile package
```

The artifacts would be built and available in the target / folder.

### Running Tests

```sh
export LD_LIBRARY_PATH={ PATH TO YOUR LOCAL libcuvs_c.so }:$LD_LIBRARY_PATH && mvn clean test
```
Expand Down
56 changes: 55 additions & 1 deletion bench/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,65 @@

<groupId>com.nvidia.cuvs.lucene.benchmarks</groupId>
<artifactId>cuvs-lucene-benchmarks</artifactId>
<!--CUVS_JAVA#VERSION_UPDATE_MARKER_START--><version>26.02.0</version><!--CUVS_JAVA#VERSION_UPDATE_MARKER_END-->
<!--CUVS_LUCENE#VERSION_UPDATE_MARKER_START--><version>26.08.0</version><!--CUVS_LUCENE#VERSION_UPDATE_MARKER_END-->

@jakirkham jakirkham Jun 25, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there anywhere else that needs this change?

@bdice bdice Jun 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly needs changes in ci/release/update-version.sh?

@jameslamb jameslamb Jun 26, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there anywhere else that needs this change?

No.

$ git grep -E '26\.|25\.'
.github/workflows/build.yaml:      container_image: "rapidsai/ci-conda:26.08-cuda${{ matrix.cuda_version }}-ubuntu24.04-py3.13"
.github/workflows/pr.yaml:      container_image: "rapidsai/ci-conda:26.08-cuda${{ matrix.cuda_version }}-ubuntu24.04-py3.13"
.github/workflows/test.yaml:      container_image: "rapidsai/ci-conda:26.08-cuda${{ matrix.cuda_version }}-ubuntu24.04-py3.13"
.pre-commit-config.yaml:        rev: v1.26.1
VERSION:26.08.00
bench/pom.xml:    <!--CUVS_LUCENE#VERSION_UPDATE_MARKER_START--><version>26.08.0</version><!--CUVS_LUCENE#VERSION_UPDATE_MARKER_END-->
bench/pom.xml:            <!--CUVS_LUCENE#VERSION_UPDATE_MARKER_START--><version>26.08.0</version><!--CUVS_LUCENE#VERSION_UPDATE_MARKER_END-->
build.sh:VERSION="26.08.0" # Note: The version is updated automatically when ci/release/update-version.sh is invoked
ci/release/update-version.sh:# Strip leading 0s in versions, so e.g. '25.10.00' becomes '25.10.0'
conda/environments/all_cuda-129_arch-aarch64.yaml:- libcuvs==26.08.*
conda/environments/all_cuda-129_arch-x86_64.yaml:- libcuvs==26.08.*
conda/environments/all_cuda-132_arch-aarch64.yaml:- libcuvs==26.08.*
conda/environments/all_cuda-132_arch-x86_64.yaml:- libcuvs==26.08.*
dependencies.yaml:          - libcuvs==26.08.*
examples/README.md:docker run --rm --gpus all --pull=always --volume $PWD:$PWD --workdir $PWD -it rapidsai/ci-conda:26.08-cuda13.2.0-ubuntu24.04-py3.13
examples/README.md:mvn clean install && java -Djava.util.logging.config.file=src/main/resources/logging.properties -cp target/examples-26.08.0-jar-with-merged-services.jar com.nvidia.cuvs.lucene.examples.AcceleratedHnswExample
examples/README.md:mvn clean install && java -Djava.util.logging.config.file=src/main/resources/logging.properties -cp target/examples-26.08.0-jar-with-merged-services.jar com.nvidia.cuvs.lucene.examples.IndexAndSearchonGPUExample
examples/pom.xml:  <!--CUVS_LUCENE#VERSION_UPDATE_MARKER_START--><version>26.08.0</version><!--CUVS_LUCENE#VERSION_UPDATE_MARKER_END-->
examples/pom.xml:      <!--CUVS_LUCENE#VERSION_UPDATE_MARKER_START--><version>26.08.0</version><!--CUVS_LUCENE#VERSION_UPDATE_MARKER_END-->
pom.xml:  <!--CUVS_LUCENE#VERSION_UPDATE_MARKER_START--><version>26.08.0</version><!--CUVS_LUCENE#VERSION_UPDATE_MARKER_END-->
pom.xml:      <!--CUVS_LUCENE#VERSION_UPDATE_MARKER_START--><version>26.08.0</version><!--CUVS_LUCENE#VERSION_UPDATE_MARKER_END-->
src/main/java/com/nvidia/cuvs/lucene/CuVS2510GPUSearchCodec.java: * @since 25.10
src/main/java/com/nvidia/cuvs/lucene/CuVS2510GPUVectorsFormat.java: * @since 25.10
src/main/java/com/nvidia/cuvs/lucene/CuVS2510GPUVectorsReader.java: * @since 25.10
src/main/java/com/nvidia/cuvs/lucene/CuVS2510GPUVectorsWriter.java: * @since 25.10
src/main/java/com/nvidia/cuvs/lucene/FilterCuVSServiceProvider.java: * @since 25.10
src/main/java/com/nvidia/cuvs/lucene/GPUBuiltHnswGraph.java: * @since 25.10
src/main/java/com/nvidia/cuvs/lucene/GPUFieldWriter.java: * @since 25.10
src/main/java/com/nvidia/cuvs/lucene/GPUIndex.java: * @since 25.10
src/main/java/com/nvidia/cuvs/lucene/GPUKnnFloatVectorQuery.java: * @since 25.10
src/main/java/com/nvidia/cuvs/lucene/GPUPerLeafCuVSKnnCollector.java: * @since 25.10
src/main/java/com/nvidia/cuvs/lucene/IndexInputInputStream.java: * @since 25.10
src/main/java/com/nvidia/cuvs/lucene/IndexOutputOutputStream.java: * @since 25.10
src/main/java/com/nvidia/cuvs/lucene/Lucene101AcceleratedHNSWCodec.java: * @since 25.10
src/main/java/com/nvidia/cuvs/lucene/Lucene99AcceleratedHNSWVectorsFormat.java: * @since 25.10
src/main/java/com/nvidia/cuvs/lucene/Lucene99AcceleratedHNSWVectorsWriter.java: * @since 25.10
src/main/java/com/nvidia/cuvs/lucene/LuceneAcceleratedHNSWBinaryQuantizedCodec.java: * @since 26.02
src/main/java/com/nvidia/cuvs/lucene/LuceneAcceleratedHNSWBinaryQuantizedVectorsFormat.java: * @since 26.02
src/main/java/com/nvidia/cuvs/lucene/LuceneAcceleratedHNSWBinaryQuantizedVectorsWriter.java: * @since 26.02
src/main/java/com/nvidia/cuvs/lucene/LuceneAcceleratedHNSWScalarQuantizedCodec.java: * @since 26.02
src/main/java/com/nvidia/cuvs/lucene/LuceneAcceleratedHNSWScalarQuantizedVectorsFormat.java: * @since 26.02
src/main/java/com/nvidia/cuvs/lucene/LuceneAcceleratedHNSWScalarQuantizedVectorsWriter.java: * @since 26.02
src/main/java/com/nvidia/cuvs/lucene/LuceneProvider.java: * @since 25.12
src/main/java/com/nvidia/cuvs/lucene/ThreadLocalCuVSResourcesProvider.java: * @since 26.02
src/main/java/com/nvidia/cuvs/lucene/Utils.java: * @since 25.10
src/main/java/com/nvidia/cuvs/lucene/package-info.java: * @since 25.10
src/test/java/com/nvidia/cuvs/lucene/TestBackCompat.java: * @since 25.12

Possibly needs changes in ci/release/update-version.sh?

I don't think so. The file was covered already, the pattern was just wrong.

https://github.com/rapidsai/cuvs-lucene/blob/5e438035c673a33fef05517718439a3b8e6a36f2/ci/release/update-version.sh#L109-L11

bash ./ci/release/update-version.sh '26.10.00' --run-context=main
git grep -E '25\.|26\.'
# (same result as above, but with all 26.08 changed to 26.10)

<packaging>jar</packaging>

<name>cuvs-lucene-benchmarks</name>

<repositories>
<!-- Prefer Google Cloud mirror, which has higher rate limits -->
<repository>
<id>gcs-maven-central-mirror</id>
<name>GCS Maven Central mirror</name>
<url>https://maven-central.storage-download.googleapis.com/maven2/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
<!-- Fall back to Maven upstream for packages not found in GCS mirror -->
<repository>
<id>central</id>
<name>Maven Central</name>
<url>https://repo.maven.apache.org/maven2</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>

<pluginRepositories>
<!-- Prefer Google Cloud mirror, which has higher rate limits -->
<pluginRepository>
<id>gcs-maven-central-mirror</id>
<name>GCS Maven Central mirror</name>
<url>https://maven-central.storage-download.googleapis.com/maven2/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
<!-- Fall back to Maven upstream for packages not found in GCS mirror -->
<pluginRepository>
<id>central</id>
<name>Maven Plugin Repository</name>
<url>https://repo.maven.apache.org/maven2</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>

<dependencies>
<dependency>
<groupId>org.openjdk.jmh</groupId>
Expand Down
6 changes: 6 additions & 0 deletions examples/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,33 +3,39 @@
This maven project contains basic examples that showcase how `cuvs-lucene` can be used.

## Prerequisites

- [Docker](https://www.docker.com/)
- [Nvidia Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
- A machine with an Nvidia GPU

## Steps

If you are currently in this directory (and to be in the `cuvs-lucene's` root directory) do:

```sh
cd ..
```

Then do:

```sh
docker run --rm --gpus all --pull=always --volume $PWD:$PWD --workdir $PWD -it rapidsai/ci-conda:26.08-cuda13.2.0-ubuntu24.04-py3.13
```

Inside the docker container (and in the `cuvs-lucene's` root directory) do:

```sh
./ci/build_java.sh && conda activate java && cd examples
```

To run Accelerated HNSW example do:

```sh
mvn clean install && java -Djava.util.logging.config.file=src/main/resources/logging.properties -cp target/examples-26.08.0-jar-with-merged-services.jar com.nvidia.cuvs.lucene.examples.AcceleratedHnswExample
```

To run the Index and Search on GPU example do:

```sh
mvn clean install && java -Djava.util.logging.config.file=src/main/resources/logging.properties -cp target/examples-26.08.0-jar-with-merged-services.jar com.nvidia.cuvs.lucene.examples.IndexAndSearchonGPUExample
```
57 changes: 53 additions & 4 deletions examples/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,59 @@
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
</properties>

<dependencyManagement>
<dependencies>
</dependencies>
</dependencyManagement>
<repositories>
<!-- Prefer Google Cloud mirror, which has higher rate limits -->
Comment thread
jameslamb marked this conversation as resolved.
<repository>
<id>gcs-maven-central-mirror</id>
<name>GCS Maven Central mirror</name>
<url>https://maven-central.storage-download.googleapis.com/maven2/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
<!-- Fall back to Maven upstream for packages not found in GCS mirror -->
<repository>
<id>central</id>
<name>Maven Central</name>
<url>https://repo.maven.apache.org/maven2</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>

<pluginRepositories>
<!-- Prefer Google Cloud mirror, which has higher rate limits -->
<pluginRepository>
<id>gcs-maven-central-mirror</id>
<name>GCS Maven Central mirror</name>
<url>https://maven-central.storage-download.googleapis.com/maven2/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
<!-- Fall back to Maven upstream for packages not found in GCS mirror -->
<pluginRepository>
<id>central</id>
<name>Maven Plugin Repository</name>
<url>https://repo.maven.apache.org/maven2</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>

<dependencies>
<dependency>
Expand Down
54 changes: 54 additions & 0 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,60 @@
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
</properties>

<repositories>
<!-- Prefer Google Cloud mirror, which has higher rate limits -->
<repository>
<id>gcs-maven-central-mirror</id>
<name>GCS Maven Central mirror</name>
<url>https://maven-central.storage-download.googleapis.com/maven2/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
<!-- Fall back to Maven upstream for packages not found in GCS mirror -->
<repository>
<id>central</id>
<name>Maven Central</name>
<url>https://repo.maven.apache.org/maven2</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>

<pluginRepositories>
<!-- Prefer Google Cloud mirror, which has higher rate limits -->
<pluginRepository>
<id>gcs-maven-central-mirror</id>
<name>GCS Maven Central mirror</name>
<url>https://maven-central.storage-download.googleapis.com/maven2/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
<!-- Fall back to Maven upstream for packages not found in GCS mirror -->
<pluginRepository>
<id>central</id>
<name>Maven Plugin Repository</name>
<url>https://repo.maven.apache.org/maven2</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>

<dependencies>
<dependency>
<groupId>org.apache.lucene</groupId>
Expand Down
Loading