-
Notifications
You must be signed in to change notification settings - Fork 1.2k
feat(integrations): add support for meilisearch #6713
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
qdequele
wants to merge
18
commits into
NangoHQ:master
Choose a base branch
from
qdequele:feat/meilisearch-integration
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
18 commits
Select commit
Hold shift + click to select a range
eaee46d
docs: add Meilisearch integration design spec
qdequele 3f89c18
docs: add Meilisearch integration implementation plan
qdequele 3a98798
feat(providers): add Meilisearch (API_KEY) provider
qdequele 373ef79
chore(meilisearch): scaffold zero-yaml integration project
qdequele 69bf73c
docs(meilisearch): add integration docs, logo, and nav entry
qdequele 0b0a0bc
feat(meilisearch): add tenant-token HS256 signer
qdequele 718bd49
feat(meilisearch): add shared zod schemas
qdequele 8152669
feat(meilisearch): add document and tenant-token actions
qdequele dd6eede
refactor(meilisearch): encode path params and add negative schema test
qdequele bd8e538
fix(meilisearch): address code-review findings
qdequele dd66d93
chore(meilisearch): add .nango marker and gitignore for zero-yaml CLI
qdequele 32caa28
docs: remove meilisearch implementation plan
qdequele 01280c5
docs: remove meilisearch design spec
qdequele 49eaede
fix(meilisearch): address PR review findings
qdequele 42f23e5
fix(meilisearch): validate action input at runtime via zodValidateInput
qdequele 9df08f3
fix(meilisearch): declare open input schemas for search/fetch passthr…
qdequele eb72a4d
chore(meilisearch): use official logo
qdequele 34e112e
chore(docs): regenerate LLM indexes after rebase
qdequele File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| --- | ||
| title: Meilisearch | ||
| sidebarTitle: Meilisearch | ||
| --- | ||
|
|
||
| import Overview from "/snippets/overview.mdx" | ||
|
|
||
| <Overview /> | ||
|
|
||
| ## Access requirements | ||
| | Pre-Requisites | Status | Comment| | ||
| | - | - | - | | ||
| | Paid dev account | ❌ | Meilisearch is open source; Cloud has a free tier. | | ||
| | Paid test account | ❌ | | | ||
| | Partnership | ❌ | | | ||
| | App review | ❌ | | | ||
| | Security audit | ❌ | | | ||
|
|
||
| ## Connecting to Meilisearch | ||
|
|
||
| Meilisearch uses API-key authentication. When creating a connection provide: | ||
|
|
||
| - **Instance URL** — your Meilisearch host including scheme, without a trailing slash, e.g. `https://ms-xxxx.meilisearch.io` or `http://localhost:7700`. | ||
| - **API Key** — a Meilisearch API key (not the master key). The Default Admin API Key works out of the box. A custom key must cover the actions you plan to use (`search`, `documents.*`, `tasks.get`, plus `version` for connection verification and `keys.get` for automatic tenant-token uid resolution). | ||
|
|
||
| ## Pre-built actions | ||
|
|
||
| | Action | Description | | ||
| | - | - | | ||
| | `search-documents` | Search documents in an index. | | ||
| | `get-documents` | Fetch documents from an index, optionally filtered. | | ||
| | `add-documents` | Add or replace documents (batch). | | ||
| | `update-documents` | Add or partially update documents (batch). | | ||
| | `delete-documents` | Delete documents by ids or filter. | | ||
| | `get-task` | Poll the status of an async write task. | | ||
| | `generate-tenant-token` | Mint a scoped, signed tenant-token JWT carrying per-index ACL search rules. | | ||
|
|
||
| ## Tenant tokens & ACL | ||
|
|
||
| `generate-tenant-token` signs an HS256 JWT with the connection's API key. The token carries `searchRules` (per-index filters) that scope what an end user can search; the signing key's ACL bounds the token. Example input: | ||
|
|
||
| ```json | ||
| { | ||
| "searchRules": { "medical_records": { "filter": "patient_id = 42" } }, | ||
| "expiresInSeconds": 3600 | ||
| } | ||
| ``` | ||
|
|
||
| Notes: | ||
|
|
||
| - Tokens expire after **1 hour** by default. Set `expiresInSeconds` or `expiresAt` (epoch seconds) to change this; `expiresAt` takes precedence when both are provided. | ||
| - A rule value can also be `null` (no restriction), e.g. `{ "searchRules": { "*": null } }`. | ||
| - If the connection's key lacks the `keys.get` action, pass `apiKeyUid` explicitly (find it via `GET /keys` with an authorized key). Master keys cannot sign tenant tokens. | ||
|
|
||
| ## Useful links | ||
|
|
||
| - [Meilisearch API keys](https://www.meilisearch.com/docs/reference/api/keys) | ||
| - [Tenant tokens](https://www.meilisearch.com/docs/learn/security/basic_security) | ||
| - [Documents API](https://www.meilisearch.com/docs/reference/api/documents) | ||
|
|
||
| <Note>Contribute improvements by [editing this page](https://github.com/nangohq/nango/tree/master/docs/integrations/all/meilisearch.mdx)</Note> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| dist/ | ||
| build/ | ||
| node_modules/ | ||
| .env | ||
| .nango/* | ||
| !.nango/.gitkeep |
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| import './meilisearch/actions/generate-tenant-token.js'; | ||
| import './meilisearch/actions/search-documents.js'; | ||
| import './meilisearch/actions/get-documents.js'; | ||
| import './meilisearch/actions/add-documents.js'; | ||
| import './meilisearch/actions/update-documents.js'; | ||
| import './meilisearch/actions/delete-documents.js'; | ||
| import './meilisearch/actions/get-task.js'; |
31 changes: 31 additions & 0 deletions
31
integrations/meilisearch/meilisearch/actions/add-documents.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| import { createAction } from 'nango'; | ||
| import * as z from 'zod'; | ||
|
|
||
| import { enqueuedTaskSchema, meiliDocumentSchema } from '../lib/schemas.js'; | ||
|
|
||
| const inputSchema = z.object({ | ||
| indexUid: z.string(), | ||
| documents: z.array(meiliDocumentSchema).min(1), | ||
| primaryKey: z.string().optional() | ||
| }); | ||
|
|
||
| const action = createAction({ | ||
| description: 'Add or replace documents in a Meilisearch index (batch). Returns the enqueued task.', | ||
| version: '1.0.0', | ||
| endpoint: { method: 'POST', path: '/meilisearch/documents', group: 'Documents' }, | ||
| input: inputSchema, | ||
| output: enqueuedTaskSchema, | ||
|
|
||
| exec: async (nango, rawInput) => { | ||
| // Declared input schemas are not enforced at runtime; validate explicitly. | ||
| const { data: input } = await nango.zodValidateInput({ zodSchema: inputSchema, input: rawInput }); | ||
| const res = await nango.post({ | ||
| endpoint: `/indexes/${encodeURIComponent(input.indexUid)}/documents`, | ||
| data: input.documents, | ||
| ...(input.primaryKey ? { params: { primaryKey: input.primaryKey } } : {}) | ||
| }); | ||
| return res.data; | ||
| } | ||
| }); | ||
|
|
||
| export default action; | ||
38 changes: 38 additions & 0 deletions
38
integrations/meilisearch/meilisearch/actions/delete-documents.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| import { createAction } from 'nango'; | ||
| import * as z from 'zod'; | ||
|
|
||
| import { enqueuedTaskSchema, filterSchema } from '../lib/schemas.js'; | ||
|
|
||
| const inputSchema = z.object({ | ||
| indexUid: z.string(), | ||
| ids: z | ||
| .array(z.union([z.string(), z.number()])) | ||
| .min(1) | ||
| .optional(), | ||
| filter: filterSchema.optional() | ||
| }); | ||
|
|
||
| const action = createAction({ | ||
| description: 'Delete documents from a Meilisearch index by ids or by filter (exactly one must be provided). Returns the enqueued task.', | ||
| version: '1.0.0', | ||
| endpoint: { method: 'POST', path: '/meilisearch/documents/delete', group: 'Documents' }, | ||
| input: inputSchema, | ||
| output: enqueuedTaskSchema, | ||
|
|
||
| exec: async (nango, rawInput) => { | ||
| // Declared input schemas are not enforced at runtime; validate explicitly. | ||
| const { data: input } = await nango.zodValidateInput({ zodSchema: inputSchema, input: rawInput }); | ||
| const hasIds = input.ids !== undefined; | ||
| const hasFilter = input.filter !== undefined; | ||
| if (hasIds === hasFilter) { | ||
| throw new nango.ActionError({ message: 'Provide exactly one of "ids" or "filter".' }); | ||
| } | ||
|
|
||
| const res = hasIds | ||
| ? await nango.post({ endpoint: `/indexes/${encodeURIComponent(input.indexUid)}/documents/delete-batch`, data: input.ids }) | ||
| : await nango.post({ endpoint: `/indexes/${encodeURIComponent(input.indexUid)}/documents/delete`, data: { filter: input.filter } }); | ||
| return res.data; | ||
| } | ||
| }); | ||
|
|
||
| export default action; |
92 changes: 92 additions & 0 deletions
92
integrations/meilisearch/meilisearch/actions/generate-tenant-token.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| import { createAction } from 'nango'; | ||
| import * as z from 'zod'; | ||
|
|
||
| import { searchRulesSchema } from '../lib/schemas.js'; | ||
| import { generateTenantToken } from '../lib/tenant-token.js'; | ||
|
|
||
| const DEFAULT_TTL_SECONDS = 3600; | ||
| const KEYS_PAGE_SIZE = 100; | ||
| const MAX_KEY_PAGES = 20; | ||
|
|
||
| const inputSchema = z.object({ | ||
| searchRules: searchRulesSchema, | ||
| expiresAt: z.number().optional().describe('Expiry as epoch seconds. Takes precedence over expiresInSeconds.'), | ||
| expiresInSeconds: z.number().optional().describe('Expiry as a duration from now. Defaults to 3600 (1 hour) when neither field is set.'), | ||
| apiKeyUid: z | ||
| .string() | ||
| .optional() | ||
| .describe('The uid of the signing API key. When omitted, it is resolved by listing keys, which requires the keys.get action.') | ||
| }); | ||
|
|
||
| const output = z.object({ | ||
| token: z.string(), | ||
| expiresAt: z.number() | ||
| }); | ||
|
|
||
| const action = createAction({ | ||
| description: | ||
| 'Generate a Meilisearch tenant token: a scoped, signed search JWT carrying per-index ACL rules. Expires after 1 hour unless expiresAt or expiresInSeconds is set.', | ||
| version: '1.0.0', | ||
| endpoint: { method: 'POST', path: '/meilisearch/tenant-token', group: 'Tenant Tokens' }, | ||
| input: inputSchema, | ||
| output, | ||
|
|
||
| exec: async (nango, rawInput) => { | ||
| // Declared input schemas are not enforced at runtime; validate explicitly. | ||
| // The strict rule schema matters here: Meilisearch ignores unknown rule keys, | ||
| // so a typo'd key would silently broaden a signed token's access. | ||
| const { data: input } = await nango.zodValidateInput({ zodSchema: inputSchema, input: rawInput }); | ||
| if (Object.keys(input.searchRules).length === 0) { | ||
| throw new nango.ActionError({ message: 'searchRules must define at least one index rule.' }); | ||
| } | ||
|
|
||
| const credentials = await nango.getToken(); | ||
| if (typeof credentials === 'string' || !('apiKey' in credentials)) { | ||
| throw new nango.ActionError({ message: 'Meilisearch connection must use API_KEY auth to mint tenant tokens.' }); | ||
| } | ||
| const apiKey = credentials.apiKey; | ||
|
|
||
| let apiKeyUid = input.apiKeyUid; | ||
| if (!apiKeyUid) { | ||
| // Resolve the key's uid by listing keys and matching locally: GET /keys/{key} | ||
| // would put the raw key in the URL path, which proxy/access logs record unredacted. | ||
| try { | ||
| for (let page = 0; page < MAX_KEY_PAGES && !apiKeyUid; page++) { | ||
| const res = await nango.get<{ results: { key: string; uid: string }[]; total: number }>({ | ||
| endpoint: '/keys', | ||
| params: { limit: String(KEYS_PAGE_SIZE), offset: String(page * KEYS_PAGE_SIZE) } | ||
| }); | ||
| apiKeyUid = res.data.results.find((k) => k.key === apiKey)?.uid; | ||
| if ((page + 1) * KEYS_PAGE_SIZE >= res.data.total) { | ||
| break; | ||
| } | ||
| } | ||
| } catch { | ||
| throw new nango.ActionError({ | ||
| message: | ||
| 'Could not list keys to resolve the API key uid. This requires a key with the keys.get action. Either connect with a key that has keys.get, or pass apiKeyUid explicitly.' | ||
| }); | ||
| } | ||
| if (!apiKeyUid) { | ||
| throw new nango.ActionError({ | ||
| message: | ||
| 'The connection API key was not found among the instance keys (the master key cannot sign tenant tokens). Connect with a regular API key, or pass apiKeyUid explicitly.' | ||
| }); | ||
| } | ||
| } | ||
|
|
||
| const nowSeconds = Math.floor(Date.now() / 1000); | ||
| const expiresAt = input.expiresAt ?? nowSeconds + (input.expiresInSeconds ?? DEFAULT_TTL_SECONDS); | ||
|
|
||
| const token = generateTenantToken({ | ||
| apiKey, | ||
| apiKeyUid, | ||
| searchRules: input.searchRules, | ||
| expiresAt | ||
| }); | ||
|
|
||
| return { token, expiresAt }; | ||
| } | ||
| }); | ||
|
|
||
| export default action; |
42 changes: 42 additions & 0 deletions
42
integrations/meilisearch/meilisearch/actions/get-documents.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| import { createAction } from 'nango'; | ||
| import * as z from 'zod'; | ||
|
|
||
| import { filterSchema, meiliDocumentSchema } from '../lib/schemas.js'; | ||
|
|
||
| // Loose: extra keys are forwarded to Meilisearch so callers can use any | ||
| // supported fetch param without a schema change. | ||
| const inputSchema = z.looseObject({ | ||
| indexUid: z.string(), | ||
| ids: z.array(z.union([z.string(), z.number()])).optional(), | ||
| filter: filterSchema.optional(), | ||
| fields: z.array(z.string()).optional(), | ||
| limit: z.number().optional(), | ||
| offset: z.number().optional() | ||
| }); | ||
|
|
||
| const output = z | ||
| .object({ | ||
| results: z.array(meiliDocumentSchema), | ||
| total: z.number(), | ||
| limit: z.number(), | ||
| offset: z.number() | ||
| }) | ||
| .catchall(z.unknown()); | ||
|
|
||
| const action = createAction({ | ||
| description: 'Fetch documents from a Meilisearch index, optionally filtered.', | ||
| version: '1.0.0', | ||
| endpoint: { method: 'POST', path: '/meilisearch/documents/fetch', group: 'Documents' }, | ||
| input: inputSchema, | ||
| output, | ||
|
|
||
| exec: async (nango, rawInput) => { | ||
| // Declared input schemas are not enforced at runtime; validate explicitly. | ||
| const { data: input } = await nango.zodValidateInput({ zodSchema: inputSchema, input: rawInput }); | ||
| const { indexUid, ...body } = input; | ||
| const res = await nango.post({ endpoint: `/indexes/${encodeURIComponent(indexUid)}/documents/fetch`, data: body }); | ||
| return res.data; | ||
| } | ||
| }); | ||
|
|
||
| export default action; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| import { createAction } from 'nango'; | ||
| import * as z from 'zod'; | ||
|
|
||
| const inputSchema = z.object({ | ||
| taskUid: z.number() | ||
| }); | ||
|
|
||
| const output = z | ||
| .object({ | ||
| uid: z.number(), | ||
| indexUid: z.string().nullable(), | ||
| status: z.string(), | ||
| type: z.string(), | ||
| error: z.unknown().nullable().optional(), | ||
| enqueuedAt: z.string(), | ||
| startedAt: z.string().nullable().optional(), | ||
| finishedAt: z.string().nullable().optional() | ||
| }) | ||
| .catchall(z.unknown()); | ||
|
|
||
| const action = createAction({ | ||
| description: 'Fetch the status of a Meilisearch async task by its uid.', | ||
| version: '1.0.0', | ||
| endpoint: { method: 'GET', path: '/meilisearch/tasks', group: 'Tasks' }, | ||
| input: inputSchema, | ||
| output, | ||
|
|
||
| exec: async (nango, rawInput) => { | ||
| // Declared input schemas are not enforced at runtime; validate explicitly. | ||
| const { data: input } = await nango.zodValidateInput({ zodSchema: inputSchema, input: rawInput }); | ||
| const res = await nango.get({ endpoint: `/tasks/${input.taskUid}` }); | ||
| return res.data; | ||
| } | ||
| }); | ||
|
|
||
| export default action; |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.