Skip to content

Commit

Permalink
Update readme
Browse files Browse the repository at this point in the history
Signed-off-by: Jay Wang <[email protected]>
  • Loading branch information
xiaohk committed Feb 9, 2024
1 parent ecf33c7 commit 17f3fcf
Show file tree
Hide file tree
Showing 6 changed files with 138 additions and 9 deletions.
4 changes: 2 additions & 2 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
MIT License

Copyright (c) 2023, Jay Wang and Polo Chau
Copyright (c) 2024, Jay Wang and Polo Chau

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand All @@ -18,4 +18,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
SOFTWARE.
120 changes: 119 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,119 @@
# Matmul
# MeMemo <a href="https://poloclub.github.io/mememo/"><img align="right" src="./examples/rag-playground/src/images/icon-logo.svg" height="38"></img></a>

[![build](https://github.com/poloclub/mememo/actions/workflows/build.yml/badge.svg)](https://github.com/poloclub/mememo/actions/workflows/build.yml)
[![npm](https://img.shields.io/npm/v/mememo?color=orange)](https://www.npmjs.com/package/mememo)
[![license](https://img.shields.io/badge/License-MIT-blue)](https://github.com/poloclub/mememo/blob/main/LICENSE)

<!-- [![arxiv badge](https://img.shields.io/badge/arXiv-2209.09227-red)](https://arxiv.org/abs/2303.09545)
[![DOI:10.1145/3543873.3587362](https://img.shields.io/badge/DOI-10.1145/3543873.3587362-blue)](https://doi.org/10.1145/3543873.3587362) -->

A JavaScript library that brings vector search and RAG to your browser!

<table>
<tr>
<td colspan="3"><a href="https://poloclub.github.io/mememo"><img src='https://i.imgur.com/4cDZQSz.png' width="100%"></a></td>
</tr>
<tr></tr>
<tr>
<td><a href="https://poloclub.github.io/mememo/?dataset=paper">🤖 ML Paper Reviewer</a></td>
<td><a href="https://poloclub.github.io/mememo/?dataset=diffusiondb">🌠 Prompt Enhancer</a></td>
<td><a href="https://poloclub.github.io/mememo/?dataset=accident">🌱 Responsible AI Assistant</a></td>
</tr>
</table>

## What is MeMemo?

MeMemo is JavaScript library that adapts the state-of-the-art approximate nearest neighbor search technique HNSW to browser environments.
Developed with modern and native Web technologies, such as IndexedDB and Web Workers, our toolkit leverages client-side hardware capabilities to enable researchers and developers to efficiently search through millions of high-dimensional vectors in browsers.
MeMemo enables exciting new design and research opportunities, such as private and personalized content creation and interactive prototyping, as demonstrated in our example application RAG Playground.✨

## Getting Started

### Installation

MeMemo supports both browser and Node.js environments. To install MeMemo, you can use `npm`:

```bash
npm install mememo
```

### Vector Search and Storage in Browsers

Then, you can create a vector index and do an approximate nearest neighbor search through two functions:

```typescript
// Import the HNSW class from the MeMemo module
import { HNSW } from 'mememo';

// Creating a new index
const index = new HNSW({ distanceFunction: 'cosine' });

// Inserting elements into our index in batches
let keys: string[];
let values: number[][];
await index.bulkInsert(keys, values);

// Find k-nearest neighbors
let query: number[];
const { keys, distances } = await index.query(query, k);
```

## Developing MeMemo

Clone or download this repository:

```bash
git clone [email protected]:poloclub/mememo.git
```

Install the dependencies:

```bash
npm install
```

Use Vitest for unit testing:

```
npm run test
```

## Developing the RAG Playground Examples

Clone or download this repository:

```bash
git clone [email protected]:poloclub/mememo.git
```

Navigate to the example folder:

```bash
cd ./examples/rag-playground
```

Install the dependencies:

```bash
npm install
```

Then run Loan Explainer:

```
npm run dev
```

Navigate to localhost:3000. You should see three Explainers running in your browser :)

## Credits

MeMemo is created by <a href='https://zijie.wang/' target='_blank'>Jay Wang</a> and <a href='' target='_blank'>Polo Chau</a>.

## License

The software is available under the [MIT License](https://github.com/poloclub/mememo/blob/main/LICENSE).

## Contact

If you have any questions, feel free to [open an issue](https://github.com/poloclub/mememo/issues/new) or contact [Jay Wang](https://zijie.wang).
1 change: 1 addition & 0 deletions examples/rag-playground/README.md
4 changes: 4 additions & 0 deletions examples/rag-playground/src/components/app/app.css
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,10 @@ button {
width: 0px;
}

a.tab {
text-decoration: none;
}

.tab {
position: relative;
display: inline-flex;
Expand Down
6 changes: 6 additions & 0 deletions examples/rag-playground/src/components/app/app.ts
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,12 @@ export class MememoRagPlayground extends LitElement {
3k
</button>
</div>
<div class="splitter"></div>
<a class="tab" href="https://github.com/poloclub/mememo"
>Add My Documents</a
>
</div>
</div>
Expand Down
12 changes: 6 additions & 6 deletions examples/rag-playground/src/components/playground/playground.ts
Original file line number Diff line number Diff line change
Expand Up @@ -83,8 +83,8 @@ const datasets: Record<Dataset, DatasetInfo> = {
},

[Dataset.arXiv120k]: {
indexURL: STORE_ENDPOINT + 'ml-arxiv-papers-index-120k.json.gzip',
dataURL: STORE_ENDPOINT + 'ml-arxiv-papers-120k.ndjson.gzip',
indexURL: REMOTE_ENDPOINT + 'ml-arxiv-papers-index-120k.json.gzip',
dataURL: REMOTE_ENDPOINT + 'ml-arxiv-papers-120k.ndjson.gzip',
datasetName: 'ml-arxiv-papers-120k',
datasetNameDisplay: 'ML arXiv Abstracts (120k)'
},
Expand All @@ -104,15 +104,15 @@ const datasets: Record<Dataset, DatasetInfo> = {
},

[Dataset.DiffusionDB500k]: {
// indexURL: '/data/diffusiondb-prompts-index-1m.json.gzip',
dataURL: '/data/diffusiondb-prompt-500k.ndjson.gzip',
indexURL: REMOTE_ENDPOINT + 'diffusiondb-prompt-index-500k.json.gzip',
dataURL: REMOTE_ENDPOINT + 'diffusiondb-prompt-500k.ndjson.gzip',
datasetName: 'diffusiondb-prompts-500k',
datasetNameDisplay: 'DiffusionDB Prompts (500k)'
},

[Dataset.DiffusionDB1m]: {
// indexURL: '/data/diffusiondb-prompts-index-1m.json.gzip',
dataURL: '/data/diffusiondb-prompt-1m.ndjson.gzip',
indexURL: REMOTE_ENDPOINT + 'diffusiondb-prompt-index-1m.json.gzip',
dataURL: REMOTE_ENDPOINT + 'diffusiondb-prompt-1m.ndjson.gzip',
datasetName: 'diffusiondb-prompts-1m',
datasetNameDisplay: 'DiffusionDB Prompts (1M)'
},
Expand Down

0 comments on commit 17f3fcf

Please sign in to comment.