dev: wait for port free in dev-reload; http: append startup audit line when writing pid

docs/integrations/pi.md

@@ -0,0 +1,43 @@
+# PI integration (example)
+
+This project ships a small HTTP server that exposes engram's query, learn,
+and context streaming endpoints. To integrate with external tooling (like
+`pi`), you can call the server from your agent process to run the same
+hook handlers engram uses internally.
+
+Quick example (requires Node 20+ and the engram HTTP server running):
+
+1. Start the engram HTTP server for your project:
+
+   - From a built installation: `engramx serve --port 7337 --project /path/to/project`
+   - Or run the server via the CLI: `node dist/cli.js serve --port 7337 --project /path/to/project`
+
+   The server writes an auth token to `~/.engram/http-server.token` on first start.
+
+2. Run the example PI client that demonstrates the flow:
+
+   ```bash
+   node examples/pi-client.js /path/to/project
+   ```
+
+   The script will:
+   - Call `POST /hook` with a `SessionStart` payload (project brief + provider warmup)
+   - Call `POST /hook` with a `UserPromptSubmit` payload (pre-query injection)
+   - POST a session summary to `/learn` so engram can persist session learnings
+
+3. Production integration suggestions:
+
+   - Call `POST /hook` with `SessionStart` at the beginning of a user session
+     to warm provider caches and inject both project and global memory.
+   - During request processing, call `POST /hook` for `UserPromptSubmit` and
+     `PreToolUse` events (or use `GET /context/stream` and `/query`) to
+     resolve context on-demand.
+   - When the request completes and you have a session summary, call `POST /learn`
+     with the summary text so engram persists the learning in its graph.
+
+Security: all HTTP endpoints require an auth token (read from
+`~/.engram/http-server.token` or supplied via `Authorization: Bearer <token>`).
+This keeps the server local-only and fail-closed.
+
+If you want, I can add a small PI-side wrapper that calls these endpoints
+from the pi harness directly (example: a `pi` skill or integration).

docs/operations/dev-reload-port-audit.md

@@ -0,0 +1,66 @@
+Dev-reload: port wait and startup audit
+
+Summary
+
+This document records the recent changes made to reduce transient EADDRINUSE errors during development reloads and to add a small startup audit line when the HTTP server writes its PID file.
+
+What was changed
+
+1) scripts/dev-reload.sh
+
+- After building the project, the script now waits (best-effort) for the configured PORT to be free before starting the server. This reduces races where nodemon spawns a new process while the old one is still closing and listening on the port.
+- Behavior is configurable via environment variables:
+  - WAIT_RETRIES (default: 16)
+  - SLEEP_INTERVAL (default: 0.5 seconds)
+- If the port remains busy after the retry loop the script proceeds anyway (non-blocking fallback) to avoid hanging CI or developer shells.
+
+2) src/server/http.ts
+
+- The writePid(projectRoot) helper still writes .engram/http-server.pid but now also appends an audit line with timestamp, pid, port, and project to two locations:
+  - <projectRoot>/.engram/http-server.start.log (project-scoped audit log)
+  - ~/.engram/http-server.log (user-level co-log)
+- Audit line format: <ISO timestamp> START pid=<pid> port=<port> project=<projectRoot>
+- Audit writes are best-effort; failures are swallowed so they don't block server startup.
+
+Why
+
+- Frequent EADDRINUSE errors were observed during rapid dev reloads. These are usually harmless (old process still closing) but they spam stderr and can hide real issues. Waiting briefly for the port to be released reduces noise and stabilizes local dev workflow.
+- The PID file used by the HUD / component-status can sometimes be missing during restart windows. The audit log makes it trivial to confirm when the server wrote the PID (and which pid/port it recorded), which aids debugging.
+
+Files modified
+
+- scripts/dev-reload.sh
+- src/server/http.ts
+
+How to test locally
+
+1) Force a dev rebuild (touch a source file and allow nodemon/dev watcher to run dev-reload)
+2) Watch the script output: you should see lines like:
+   [dev-reload] waiting for port 7337 to be free (still: <pids>) — attempt X/Y
+   [dev-reload] starting server via: node dist/cli.js ui ...
+3) After server starts, verify:
+   - cat <projectRoot>/.engram/http-server.pid → should contain the pid
+   - tail -n 50 <projectRoot>/.engram/http-server.start.log → last line is the START audit line
+   - tail -n 50 ~/.engram/http-server.log → contains the START audit line
+4) Exercise the dashboard/UI or POST a SessionStart hook to confirm the updated server responds with the injected project brief.
+
+Environment knobs
+
+- WAIT_RETRIES= number of attempts waiting for port to be free (default 16)
+- SLEEP_INTERVAL= seconds to sleep between attempts (default 0.5)
+
+Todo / Follow-ups (issues created)
+
+- [ ] Add a small integration test that verifies the audit line is written on server startup (requires test harness that can spawn the server).
+- [ ] Consider making the dev-reload script fail-hard if the port remains busy (configurable via env var) to avoid accidental double-runs in CI.
+- [ ] Rotate/trim the audit logs (project-level and user-level) to avoid unbounded disk growth.
+- [ ] Improve portability for environments without lsof (Windows) — consider a Node-based port probe helper or use netstat variations.
+- [ ] Expose the startup audit via component-status so the UI can show "Last startup at <ts> (pid)".
+
+Link to changes
+
+- Branch: feat/sessionstart-resume-auto-memory
+- Commits:
+  - dev: wait for port free in dev-reload; http: append startup audit line when writing pid
+
+If you'd like any of the follow-ups implemented now (tests, log rotation, stricter dev-reload behavior), tell me which ones and I'll implement them in follow-up commits/PRs.

examples/pi-client.js

@@ -0,0 +1,72 @@
+#!/usr/bin/env node
+/**
+ * Example PI client that demonstrates calling engram's /hook and /learn endpoints.
+ *
+ * Usage: node examples/pi-client.js /path/to/project
+ */
+
+import { readFileSync } from "node:fs";
+import { homedir } from "node:os";
+import { join } from "node:path";
+
+const PORT = process.env.ENGRAM_HTTP_PORT ?? 7337;
+const TOKEN_PATH = join(homedir(), ".engram", "http-server.token");
+
+function readToken() {
+  try {
+    const t = readFileSync(TOKEN_PATH, "utf-8").trim();
+    return t;
+  } catch (err) {
+    console.error("Failed to read token at", TOKEN_PATH, err?.message ?? err);
+    process.exit(1);
+  }
+}
+
+async function postHook(payload) {
+  const resp = await fetch(`http://127.0.0.1:${PORT}/hook`, {
+    method: "POST",
+    headers: {
+      Authorization: `Bearer ${readToken()}`,
+      "Content-Type": "application/json",
+    },
+    body: JSON.stringify(payload),
+  });
+  if (resp.status === 204) return null;
+  return await resp.json();
+}
+
+async function postLearn(content, file) {
+  const resp = await fetch(`http://127.0.0.1:${PORT}/learn`, {
+    method: "POST",
+    headers: {
+      Authorization: `Bearer ${readToken()}`,
+      "Content-Type": "application/json",
+    },
+    body: JSON.stringify({ content, file }),
+  });
+  if (resp.status >= 400) {
+    const txt = await resp.text();
+    throw new Error(`learn failed: ${txt}`);
+  }
+  return await resp.json();
+}
+
+const projectRootArg = process.argv[2] ?? process.cwd();
+
+(async () => {
+  console.log("Project root:", projectRootArg);
+  console.log("Calling SessionStart...");
+  const session = await postHook({ hook_event_name: "SessionStart", cwd: projectRootArg, source: "startup" });
+  console.log("SessionStart result:", session);
+
+  console.log("Calling UserPromptSubmit...");
+  const prompt = await postHook({ hook_event_name: "UserPromptSubmit", cwd: projectRootArg, prompt: "How does authentication work?" });
+  console.log("UserPromptSubmit result:", prompt);
+
+  console.log("Posting a summary to /learn...");
+  const learn = await postLearn("Session summary: example decision to prefer JWT tokens", "pi-example");
+  console.log("Learn result:", learn);
+})().catch((err) => {
+  console.error("Error", err);
+  process.exit(1);
+});