Skip to content

Use x-defang-llm with mastra #545

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
KevyVo wants to merge 15 commits into
base: main
Choose a base branch
from
Open

Use x-defang-llm with mastra #545

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
869b4bc
aws llm runnning
KevyVo Dec 6, 2025
1511381
working GCP
KevyVo Dec 6, 2025
7a61396
Refactors model provider logic and updates dependencies
KevyVo Dec 8, 2025
938eac8
Removes unused google-auth-library dependency
KevyVo Dec 8, 2025
1ce140a
Update README
KevyVo Dec 8, 2025
41bdb03
Simplifies model provider logic and updates environment variables
KevyVo Dec 9, 2025
c906edb
load in model for different enviromnents
KevyVo Dec 10, 2025
4a0f33b
BYOC compose for GCP
KevyVo Dec 10, 2025
249d3c4
Add hot reloading for docker compose dev
KevyVo Dec 10, 2025
7c66e84
update gitignore
KevyVo Dec 10, 2025
d47ca90
Update README
KevyVo Dec 10, 2025
94b8dd1
Update sample after setting env vars
KevyVo Dec 13, 2025
26b6edd
update test env
KevyVo Dec 14, 2025
144ccef
update function with doc link
KevyVo Dec 14, 2025
8eef1cc
update one click deployment
KevyVo Dec 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 16 additions & 4 deletions samples/mastra-nextjs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,16 +38,28 @@ You can access mastra-nextjs at `http://localhost:3000` once the containers are

For this sample, you will need to provide the following [configuration](https://docs.defang.io/docs/concepts/configuration). Note that if you are using the 1-click deploy option, you can set these values as secrets in your GitHub repository and the action will automatically deploy them for you.

### `GOOGLE_GENERATIVE_AI_API_KEY`

Your Google Generative AI API key for accessing the Gemini Flash model. You can get this from the [Google AI Studio](https://aistudio.google.com/).

### `POSTGRES_PASSWORD`

The password for your Postgres database. You need to set this before deploying for the first time.

_You can easily set this to a random string using `defang config set POSTGRES_PASSWORD --random`_

### `DEFANG_PROVIDER`

Set to `gcp` when deploying to Google Cloud, or `aws` when deploying to AWS.

### `DEFANG_GCP_PROJECT` (Only for GCP deployments)

The GCP project ID where you want to deploy the application. You can find this in the [GCP Console](https://console.cloud.google.com/).

### `DEFANG_GCP_LOCATION` (Only for GCP deployments)

The GCP region where you want to deploy the application. This should typically be set to `us-central1`, but you can change it to your preferred location.

### `LLM_MODEL`

The large language model to use for the AI-powered chat. This can be set to models like `anthropic.claude-3-5-sonnet-20241022-v2:0` for AWS or `gemini-2.5-flash` for Google Cloud. Here is a list of supported models for [GCP](https://ai-sdk.dev/providers/ai-sdk-providers/google-vertex#model-capabilities) and [AWS](https://ai-sdk.dev/providers/ai-sdk-providers/amazon-bedrock#model-capabilities). For AWS make sure you [request access](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html) to the model in the AWS Bedrock console.

### `DB_URL`

The [PostgreSQL database connection string](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING). This will be automatically configured when using BYOC managed database services. It should look something like this: `postgresql://[user[:password]@][netloc][:port][/dbname][?param1=value1&...]`.
Expand Down
2 changes: 2 additions & 0 deletions samples/mastra-nextjs/app/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,5 @@ yarn-error.log*
next-env.d.ts

.mastra
../.defang

4 changes: 4 additions & 0 deletions samples/mastra-nextjs/app/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,13 @@
"lint": "next lint"
},
"dependencies": {
"@ai-sdk/amazon-bedrock": "^3.0.67",
"@ai-sdk/google-vertex": "^3.0.86",
"@ai-sdk/provider": "^2.0.0",
"@assistant-ui/react": "^0.11.39",
"@assistant-ui/react-ai-sdk": "^1.1.11",
"@assistant-ui/react-markdown": "^0.11.4",
"@aws-sdk/credential-providers": "^3.946.0",
"@mastra/ai-sdk": "^0.2.7",
"@mastra/core": "^0.24.1",
"@mastra/memory": "^0.15.11",
Expand Down
Loading
Loading