docs: add runbook for failed Vercel deployment rollback

docs/operations/auto-rollback.md

@@ -1,10 +1,10 @@
 # Automated rollback on CI failure
 
-This runbook covers the automation for GitHub issue **#236**: detect a failed production deploy, revert to the last known-good version, notify Slack/Discord, and open an incident ticket.
+This runbook covers the automation for GitHub issue *#236*: detect a failed production deploy, revert to the last known-good version, notify Slack/Discord, and open an incident ticket.
 
 ## What triggers a rollback
 
-The workflow [`.github/workflows/auto-rollback.yml`](../../.github/workflows/auto-rollback.yml) runs when:
+The workflow [.github/workflows/auto-rollback.yml] (../../.github/workflows/auto-rollback.yml) runs when:
 
 - The **Deploy - Frontend to Vercel and Artifacts** workflow finishes with `failure` or `timed_out` on `main`, or
 - An operator starts **Auto-rollback on deploy failure** from the Actions UI (defaults to dry-run).
@@ -42,6 +42,32 @@ yarn ops:rollback --dry-run
 
 Or dispatch the workflow with **dry_run** enabled.
 
+## Reading Vercel deployment failure logs
+
+When a production deploy fails, the rollback automation uses Vercel's deployment status. To investigate the root cause:
+
+1. **List recent deployments** to identify the failed deployment ID:
+   ```bash
+   vercel ls --prod
+   ```
+   Or use the dashboard: https://vercel.com/dashboard → select project ↔ **Deployments**.
+
+2. **Inspect build logs** for the failed deployment. CLI:
+   ```bash
+   vercel logs <deployment-id>
+   ```
+   In the dashboard, open the deployment and choose **Build Logs**. Look for the first error (often TypeScript, Babel, or dependency errors).
+
+3. **Check runtime logs** if the deployment built but failed health checks:
+   ```bash
+   vercel logs <deployment-id> --json
+   ```
+   Search for `unhandled rejection`, `error`, `ECONNREFUSED`, or timeout messages.
+
+4. **Cross-reference environment variables** (required secrets above) to ensure the Vercel project has the same values as CI.
+
+Keep the logs with the incident ticket for later analysis.
+
 ## After a rollback
 
 1. Confirm `/api/health` and `/api/status` are green.

docs/troubleshooting.md

@@ -276,3 +276,23 @@ console.log(window.stellar);
 // Verify Web Crypto API support
 console.log(window.crypto && window.crypto.subtle ? "WebCrypto OK" : "WebCrypto Unavailable");
 ```
+
+### 8.4 Failed Vercel Deployment Rollback Runbook
+
+Use this runbook when a production deployment fails or auto-rolls back in Vercel.
+
+#### Reading Deployment Failure Logs
+
+1. Open the Vercel project dashboard and go to **Deployments**.
+2. Select the failed deployment (marked with a red **Error** badge).
+3. Click **Inspect Deployment** and open the **Build Logs** tab.
+4. Search for `Error:`, `Failed to compile`, or `Command exited with code` to find the root cause.
+5. For runtime failures after the build succeeds, open the **Runtime Logs** tab and filter by the failing deployment.
+
+#### Rolling Back to the Last Known Good Deployment
+
+1. In the **Deployments** tab, identify the most recent deployment with a green **Ready** badge.
+2. Click the overflow menu (three dots) on that deployment and choose **Promote to Production** (or **Redeploy** for the same production URL).
+3. Confirm the promotion in the dialog and wait for the promotion to complete.
+4. Verify the production URL returns HTTP 200 and the application loads in the browser.
+5. If the rollback itself fails, create a GitHub issue tagged `runbook:vercel-rollback` with the failed deployment URL, build log excerpt, and target deployment hash.

runbook.md

@@ -271,3 +271,29 @@ We apologize for the disruption and appreciate your patience while we work to fu
 
 Regards,
 [promptmint/IT Team]
+
+---
+
+# Failed Vercel Deployment Rollback
+
+Use this runbook when a Vercel deployment fails and Vercel automatically rolls back to the last successful deployment, or when you need to manually roll back a bad deployment.
+
+## Rollback Procedure
+
+1. Go to the Vercel project dashboard.
+2. Open the **Deployments** tab.
+3. Locate the failed deployment and note the error status.
+4. If Vercel did not automatically roll back, open the last successful deployment.
+5. Select **Promote to Production** to redeploy the last known-good version.
+6. Confirm the new production deployment reaches **Ready**.
+7. Verify the application health endpoint, critical user flows, and environment variables after rollback.
+
+## Reading Deployment Failure Logs
+
+1. Open the failed deployment in Vercel.
+2. Select **Logs** to view build/runtime logs.
+3. Look for `ERROR`, `Failed`, `Rolling back`, or non-zero exit codes near the failure point.
+4. For build failures, check dependency installation, environment variable resolution, and compile output.
+5. For runtime failures, inspect function logs for startup crashes, missing secrets, or invalid configuration.
+6. If the failure is intermittent, download or copy the logs immediately because Vercel retains deployment logs for a limited time.
+7. Attach the relevant log lines and deployment URL to the incident record before closing the issue.