Use x-defang-llm with mastra

.github/workflows/deploy-changed-samples.yml

@@ -79,7 +79,6 @@ jobs:
           TEST_DB_POSTGRESDB_SSL_ENABLED: ${{ secrets.TEST_DB_POSTGRESDB_SSL_ENABLED }}
           TEST_DB_POSTGRESDB_SSL_REJECT_UNAUTHORIZED: ${{ secrets.TEST_DB_POSTGRESDB_SSL_REJECT_UNAUTHORIZED }}
           TEST_GITHUB_TOKEN: ${{ secrets.TEST_GITHUB_TOKEN }}
-          TEST_GOOGLE_GENERATIVE_AI_API_KEY: ${{ secrets.TEST_GOOGLE_GENERATIVE_AI_API_KEY }}
           TEST_HASURA_GRAPHQL_ADMIN_SECRET: ${{ secrets.TEST_HASURA_GRAPHQL_ADMIN_SECRET }}
           TEST_HASURA_GRAPHQL_DATABASE_URL: ${{ secrets.TEST_HASURA_GRAPHQL_DATABASE_URL }}
           TEST_JUPYTER_TOKEN: ${{ secrets.TEST_JUPYTER_TOKEN }}
@@ -93,6 +92,7 @@ jobs:
           TEST_MODEL: ${{ secrets.TEST_MODEL }}
           TEST_MONGO_INITDB_ROOT_USERNAME: ${{ secrets.TEST_MONGO_INITDB_ROOT_USERNAME }}
           TEST_MONGO_INITDB_ROOT_PASSWORD: ${{ secrets.TEST_MONGO_INITDB_ROOT_PASSWORD }}
+          TEST_LLM_MODEL: ${{ secrets.TEST_LLM_MODEL }}
           TEST_N8N_ENCRYPTION_KEY: ${{ secrets.TEST_N8N_ENCRYPTION_KEY }}
           TEST_NC_DB: ${{ secrets.TEST_NC_DB }}
           TEST_NC_S3_ENDPOINT: ${{ secrets.TEST_NC_S3_ENDPOINT }}

samples/mastra-nextjs/.github/workflows/deploy.yaml

@@ -20,8 +20,7 @@ jobs:
       - name: Deploy
         uses: DefangLabs/[email protected]
         with:
-          config-env-vars: POSTGRES_PASSWORD GOOGLE_GENERATIVE_AI_API_KEY GITHUB_TOKEN
+          config-env-vars: POSTGRES_PASSWORD GITHUB_TOKEN
         env:
           POSTGRES_PASSWORD: ${{ secrets.POSTGRES_PASSWORD }}
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-          GOOGLE_GENERATIVE_AI_API_KEY: ${{ secrets.GOOGLE_GENERATIVE_AI_API_KEY }}

samples/mastra-nextjs/README.md

@@ -26,6 +26,14 @@ To run the application locally for development, use the development compose file
 docker compose -f compose.dev.yaml up
 ```
 
+### `GOOGLE_GENERATIVE_AI_API_KEY`
+
+When running locally with Docker Compose, you will need to set the `GOOGLE_GENERATIVE_AI_API_KEY` environment variable to your GCP API key. You can get the API key from the [Google AI Studio](https://aistudio.google.com/).
+
+### `LLM_MODEL`
+
+When running locally with Docker Compose, you are limited to the models on this list: [Google models](https://mastra.ai/models/providers/google).
+
 This will:
 
 - Start PostgreSQL with volume persistence for local development
@@ -38,19 +46,15 @@ You can access mastra-nextjs at `http://localhost:3000` once the containers are
 
 For this sample, you will need to provide the following [configuration](https://docs.defang.io/docs/concepts/configuration). Note that if you are using the 1-click deploy option, you can set these values as secrets in your GitHub repository and the action will automatically deploy them for you.
 
-### `GOOGLE_GENERATIVE_AI_API_KEY`
-
-Your Google Generative AI API key for accessing the Gemini Flash model. You can get this from the [Google AI Studio](https://aistudio.google.com/).
-
 ### `POSTGRES_PASSWORD`
 
 The password for your Postgres database. You need to set this before deploying for the first time.
 
 _You can easily set this to a random string using `defang config set POSTGRES_PASSWORD --random`_
 
-### `DB_URL`
+### `LLM_MODEL`
 
-The [PostgreSQL database connection string](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING). This will be automatically configured when using BYOC managed database services. It should look something like this: `postgresql://[user[:password]@][netloc][:port][/dbname][?param1=value1&...]`.
+The large language model to use for the AI-powered chat. This can be set to models like `anthropic.claude-3-5-sonnet-20241022-v2:0` for AWS or `gemini-2.5-flash` for Google Cloud. Here is a list of supported models for [GCP](https://ai-sdk.dev/providers/ai-sdk-providers/google-vertex#model-capabilities) and [AWS](https://ai-sdk.dev/providers/ai-sdk-providers/amazon-bedrock#model-capabilities). For AWS make sure you [request access](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html) to the model in AWS Bedrock console and for GCP make sure you have [request access](https://docs.cloud.google.com/marketplace/docs/enable-ai-models#get-ai-models) to the model in GCP Vertex AI console.
 
 ### `DB_SSL`
 
@@ -89,6 +93,8 @@ The large context window of Gemini Flash allows the agent to hold more code in m
 
 ### Defang Playground
 
+When deploying we only allow you to use the model `claude-3-5-haiku-20241022` due to resource constraints in the Playground environment. If you want to use other models, please use [Defang BYOC](https://docs.defang.io/docs/tutorials/deploy-to-your-cloud).
+
 Deploy your application to the Defang Playground by opening up your terminal and typing:
 
 ```bash

samples/mastra-nextjs/app/package.json

@@ -10,9 +10,13 @@
     "lint": "next lint"
   },
   "dependencies": {
+    "@ai-sdk/amazon-bedrock": "^3.0.67",
+    "@ai-sdk/google-vertex": "^3.0.86",
+    "@ai-sdk/provider": "^2.0.0",
     "@assistant-ui/react": "^0.11.39",
     "@assistant-ui/react-ai-sdk": "^1.1.11",
     "@assistant-ui/react-markdown": "^0.11.4",
+    "@aws-sdk/credential-providers": "^3.946.0",
     "@mastra/ai-sdk": "^0.2.7",
     "@mastra/core": "^0.24.1",
     "@mastra/memory": "^0.15.11",