Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for namespaces #12

Merged
merged 5 commits into from
May 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 68 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,42 @@
# Upstash Vector Go Client

[![Go Reference](https://pkg.go.dev/badge/github.com/upstash/vector-go.svg)](https://pkg.go.dev/github.com/upstash/vector-go)

> [!NOTE]
> **This project is in GA Stage.**
>
> The Upstash Professional Support fully covers this project. It receives regular updates, and bug fixes. The Upstash team is committed to maintaining and improving its functionality.
> The Upstash Professional Support fully covers this project. It receives regular updates, and bug fixes.
> The Upstash team is committed to maintaining and improving its functionality.

[Upstash](https://upstash.com/) Vector is a serverless vector database designed for working with vector embeddings.

This is the Go client for [Upstash](https://upstash.com/) Vector.
This is the HTTP-based Go client for [Upstash](https://upstash.com/) Vector.

## Documentation

- [**Reference Documentation**](https://upstash.com/docs/vector/overall/getstarted)

## Installation

Use `go get` to install the Upstash Vector package:
```bash
go get github.com/upstash/vector-go
```

Import the Upstash Vector package in your project:

```go
import "github.com/upstash/vector-go"
```

## Usage

In order to use this client, head out to [Upstash Console](https://console.upstash.com) and create a vector database.

### Initializing the client

There are two pieces of configuration required to use the Upstash Vector index client: an REST token and REST URL.
Find your configuration values in the console dashboard at [https://console.upstash.com/](https://console.upstash.com/).
The REST token and REST URL configurations are required to initialize an Upstash Vector index client.
Find your configuration values in the console dashboard at [Upstash Console](https://console.upstash.com/).

```go
import (
Expand Down Expand Up @@ -80,14 +94,43 @@ func main() {

Upstash vector indexes support operations for working with vector data using operations such as upsert, query, fetch, and delete.

```go
import (
"github.com/upstash/vector-go"
)

func main() {
index := vector.NewIndex("<UPSTASH_VECTOR_REST_URL>", "<UPSTASH_VECTOR_REST_TOKEN>")
}
```

Upstash Vector allows you to partition a single index into multiple isolated namespaces.

You can specify a namespace for an index client with `Namespace(ns string)` function.
When you create a `Namespace` client, all index operations executed through this client become associated with the specified namespace.

By default, the `Index` client is associated with the default namespace.

```go
import (
"github.com/upstash/vector-go"
)

func main() {
index := vector.NewIndex("<UPSTASH_VECTOR_REST_URL>", "<UPSTASH_VECTOR_REST_TOKEN>")

// Returns a new Namespace client associated with the given namespace
ns := index.Namespace("<NAMESPACE>")
```

### Upserting Vectors

All vectors upserted to index must have the same dimensions.

Upsert can be used to insert new vectors into index or to update
existing vectors.

#### Upsert many
#### Upsert Many

```go
upserts := []vector.Upsert{
Expand Down Expand Up @@ -287,3 +330,23 @@ err := index.Reset()
```go
info, err := index.Info()
```

### List Namespaces

All the names of active namespaces can be listed.

```go
namespaces, err := index.ListNamespaces()
for _, ns : range namespaces {
fmt.Println(ns)
}
```

### Delete Namespaces

A namespace can be deleted entirely if it exists.
The default namespaces cannot be deleted.

```go
err := index.Namespace("ns").DeleteNamespace()
```
22 changes: 14 additions & 8 deletions delete.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,19 @@ package vector

const deletePath = "/delete"

// Delete deletes the vector with the given id and reports
// whether the vector is deleted. If a vector with the given
// id is not found, Delete returns false.
// Delete deletes the vector with the given id in the default namespace and reports whether the vector is deleted.
// If a vector with the given id is not found, Delete returns false.
func (ix *Index) Delete(id string) (ok bool, err error) {
data, err := ix.sendBytes(deletePath, []byte(id))
return ix.deleteInternal(id, defaultNamespace)
}

// DeleteMany deletes the vectors with the given ids in the default namespace and reports how many of them are deleted.
func (ix *Index) DeleteMany(ids []string) (count int, err error) {
return ix.deleteManyInternal(ids, defaultNamespace)
}

func (ix *Index) deleteInternal(id string, ns string) (ok bool, err error) {
data, err := ix.sendBytes(buildPath(deletePath, ns), []byte(id))
if err != nil {
return
}
Expand All @@ -16,10 +24,8 @@ func (ix *Index) Delete(id string) (ok bool, err error) {
return
}

// DeleteMany deletes the vectors with the given ids and reports
// how many of them are deleted.
func (ix *Index) DeleteMany(ids []string) (count int, err error) {
data, err := ix.sendJson(deletePath, ids)
func (ix *Index) deleteManyInternal(ids []string, ns string) (count int, err error) {
data, err := ix.sendJson(buildPath(deletePath, ns), ids)
if err != nil {
return
}
Expand Down
91 changes: 49 additions & 42 deletions delete_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,49 +15,56 @@ func randomString() string {
}

func TestDelete(t *testing.T) {
client, err := newTestClient()
require.NoError(t, err)

id := randomString()
err = client.Upsert(Upsert{
Id: id,
Vector: []float32{0, 1},
})
require.NoError(t, err)

t.Run("existing id", func(t *testing.T) {
ok, err := client.Delete(id)
require.NoError(t, err)
require.True(t, ok)
})

t.Run("non existing id", func(t *testing.T) {
ok, err := client.Delete(randomString())
require.NoError(t, err)
require.False(t, ok)
})
for _, ns := range namespaces {
t.Run("namespace_"+ns, func(t *testing.T) {
client, err := newTestClientWithNamespace(ns)
require.NoError(t, err)
id := randomString()
err = client.Upsert(Upsert{
Id: id,
Vector: []float32{0, 1},
})
require.NoError(t, err)

t.Run("existing id", func(t *testing.T) {
ok, err := client.Delete(id)
require.NoError(t, err)
require.True(t, ok)
})

t.Run("non existing id", func(t *testing.T) {
ok, err := client.Delete(randomString())
require.NoError(t, err)
require.False(t, ok)
})
})
}
}

func TestDeleteMany(t *testing.T) {
client, err := newTestClient()
require.NoError(t, err)

id0 := randomString()
id1 := randomString()
id2 := randomString()
err = client.UpsertMany([]Upsert{
{
Id: id0,
Vector: []float32{0, 1},
},
{
Id: id1,
Vector: []float32{5, 10},
},
})
require.NoError(t, err)

count, err := client.DeleteMany([]string{id0, id1, id2})
require.NoError(t, err)
require.Equal(t, 2, count)
for _, ns := range namespaces {
t.Run("namespace "+ns, func(t *testing.T) {
client, err := newTestClientWithNamespace(ns)
require.NoError(t, err)

id0 := randomString()
id1 := randomString()
id2 := randomString()
err = client.UpsertMany([]Upsert{
{
Id: id0,
Vector: []float32{0, 1},
},
{
Id: id1,
Vector: []float32{5, 10},
},
})
require.NoError(t, err)

count, err := client.DeleteMany([]string{id0, id1, id2})
require.NoError(t, err)
require.Equal(t, 2, count)
})
}
}
Loading
Loading