Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flatnav python bindings #14

Merged
merged 9 commits into from
Nov 23, 2023
Merged

Flatnav python bindings #14

merged 9 commits into from
Nov 23, 2023

Conversation

BlaiseMuhirwa
Copy link
Owner

@BlaiseMuhirwa BlaiseMuhirwa commented Nov 21, 2023

Build Python bindings for the FlatNav library. Now we can use it like this:

import flatnav
from flatnav.index import index_factory
from flatnav.index import L2Index, IPIndex
from typing import Union 

def create_index(
    distance_type: str, dim: int, dataset_size: int, max_edges_per_node: int
) -> Union[L2Index, IPIndex]:
    index = index_factory(
        distance_type=distance_type,
        dim=dim,
        dataset_size=dataset_size,
        max_edges_per_node=max_edges_per_node,
        verbose=True,
    )
    if not (isinstance(index, L2Index) or isinstance(index, IPIndex)):
        raise RuntimeError("Invalid index.")

    return index

flatnav_index = create_index(distance_type="l2", dataset_size=60000, max_edges_per_node=32)

PR also does a few other things, mainly:

  • Make some installation scripts more robust across platforms (ex. setting up clang, which I've chosen to be the compiler for the C++ FlatNav library).
  • Remove the SQDistance.h header file, which was originally intended to be an implementation of low-precision quantization. An implementation for that is already in PR here.
  • Delete some flatnav-egg-info stuff that was erroneously committed previously.

@@ -60,7 +66,10 @@ echo "Using CC=${CC} and CXX=${CXX} compilers for building."

mkdir -p build
cd build && cmake \
-DCMAKE_C_COMPILER=${CC} \
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passing these here directly so that there is no confusion when someone has both GCC and clang.

CMakeLists.txt Outdated
# In Debug mode, we will also add the -g flag to enable debugging and the
# address sanitizer.
message(STATUS "Building in Debug mode")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -g -fsanitize=address")
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use address sanitizer only in debug builds.

"-funroll-loops", # Unroll loops
"-ftree-vectorize", # Vectorize where possible
],
extra_link_args=["-fopenmp"], # Link OpenMP when linking the extension
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think for now this only works on Linux!

@BlaiseMuhirwa BlaiseMuhirwa merged commit ccb5295 into main Nov 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants