working-notes.md

Working Notes

Why this file exists

This file preserves useful design context that is not formal enough for the PRD or an ADR.

Use it for:

• unresolved questions
• design ideas worth revisiting
• implementation watchouts
• future feature notes
• session-to-session continuity

Current direction

• Build a simple service catalog for engineers
• Keep the MVP significantly simpler than a traditional CMDB
• Git is the database
• Markdown with YAML frontmatter is the source of truth
• TypeScript is the main language
• Astro 6 is the chosen MVP framework
• Tailwind 4 for styling
• Production should ship as a Docker image (Kubernetes and Gitops ready)
• Local test should work directly without Docker
• Local test should be possible with Containers too.
• Production config and catalog content should live outside the built image
• As MVP minimum baseline: this catalog should have more value than Services described in Confluence
• The catalog can now merge a local source with one or more managed Git sources
• Managed Git sync is now handled by a small in-process scheduler instead of syncing on every request

Confirmed decisions already captured elsewhere

See:

• docs/prd.md
• .adr/001-choose-typescript-and-astro-for-mvp.md
• .adr/002-use-git-backed-markdown-as-source-of-truth.md
• .adr/003-deploy-as-stateless-container-with-external-catalog-path.md
• .adr/004-separate-catalog-core-from-source-ingestion-and-sync.md
• .adr/006-add-app-managed-git-checkout-for-multiple-catalog-sources.md
• .adr/007-protect-the-catalog-with-basic-auth.md
• .adr/008-add-an-in-process-git-sync-scheduler.md
• .adr/010-adopt-pino-for-application-logging.md

Important design reminders

• Do not overbuild the frontend too early
• Do not let future repository sync requirements distort the MVP
• Validation is part of the product, not just plumbing
• The catalog core should remain independent of future scheduling/sync concerns
• Request rendering should not perform Git network operations
• For managed Git, keep sync and checkout management boring and predictable
• The common Kubernetes case should stay simple: emptyDir cache, mounted SSH key, mounted known_hosts
• Basic Auth realm is fixed to servdir
• Probe endpoints should stay unauthenticated: /health/live for liveness and /health/ready for config/readiness checks
• Managed Git source UI should distinguish configured sources from failing sources, using the runtime sync status that already exists in memory

Cross-session handoff essentials

Read this section first when picking the project up in a fresh session. It exists so future agents do not need to reconstruct important context from chat history.

Verification baseline

Use this as the default verification set after UI, routing, or build-related changes:

• pnpm test
• pnpm build
• pnpm build:static

Reason:

• servdir intentionally supports both the default server runtime and a secondary static export mode
• recent work touched both UI composition and route/build behavior, so checking only one build mode is not enough

Current icon strategy

• Astro icons are wired through astro-icon in astro.config.mjs
• app code should go through src/components/ui/Icon.astro, not raw astro-icon usage spread across the codebase
• app-specific icons live as local SVGs under src/icons/
• when a UI needs semantic icons, prefer a small explicit mapping helper plus local SVGs over introducing a large external icon set

Reason:

• local SVGs are predictable and avoid fragile icon-set dependencies for a small number of domain-specific icons
• the wrapper keeps icon usage consistent and easier to change later

Catalog status card structure

The catalog status area was intentionally split into small Astro pieces for readability:

• src/components/catalog/CatalogStatusCard.astro — composition shell
• src/components/catalog/CatalogStatusTabs.astro — tab buttons
• src/components/catalog/CatalogStatusPanel.astro — accessible panel wrapper
• src/components/catalog/CatalogStatusDetailList.astro — shared detail grid
• src/components/catalog/CatalogStatusGitSourcesPopover.astro — compact Git source details popover
• src/lib/catalog-status.ts — pure labels/format/id helpers
• src/scripts/catalog-status-card.ts — tab and popover behavior

Preserve these boundaries unless there is a clear readability win. Do not collapse them back into one large .astro file.

Catalog toolbar layout

The catalog index toolbar is a single compact row (on desktop) containing:

• A search input (flex-1) on the left
• Kind filter buttons (icon-only circles, one per kind present) to the right of search
• A view/layout pill (List | Cards | Group) at the far right

A slim secondary row below shows the live count and the Browse tags link.

On mobile the row wraps naturally.

Platform pages

• /platforms index lists all known platforms derived from service platform fields, sorted alphabetically with service counts.
• /platforms/[slug] shows all services on that platform using the standard ServiceCatalogGrid.
• Platform slugs are normalized the same way as tag slugs (trim, lowercase, non-alphanumeric → -).
• "Browse platforms" link appears on the catalog index only when at least one service has a platform value.
• Data logic lives in src/lib/catalog/platform-page.ts, following the same shape as tag-page.ts.
• PlatformList.astro renders the index grid, mirroring TagCloud.astro.

Search

• Search filters by name and id, case-insensitive, as you type.
• Search state is held in catalog-grid.ts alongside kind filter and platform grouping.
• All three filters are evaluated together in a single applyFilters() call — no separate per-filter visibility logic.
• Each [data-service-kind] item carries data-service-name and data-service-id (both lowercased) for JS matching without DOM text traversal.
• Search works across both list and card views, and in flat and grouped platform modes.

Platform grouping

• A platform field (optional free string, e.g. aws-prod, on-prem, legacy-k8s, hetzner) was added to the service schema.
• The catalog index renders both a flat and a grouped version of both the list and card views.
• The platform grouping toggle (Group button) lives inside the view mode pill, separated by a thin divider — clearly a layout control, not a filter.
• Clicking the toggle switches between flat and grouped views; clicking again reverts.
• Platform section headers show the platform name and entry count.
• Entries without a platform value are grouped last under "Other".
• The kind filter and search remain functional in both flat and grouped modes; empty platform sections are hidden automatically.
• The flat view maintains the existing single-list appearance unchanged.

Service list and card conventions

• compact list rows now show a small kind icon before the stable service id
• kind icon mapping lives in:
  • src/components/catalog/CatalogKindIcon.astro
  • src/lib/catalog-kind-icon.ts
• currently supported explicit kinds are:
  • service
  • tool
  • application
  • library
  • component
  • iac
• unknown kinds are still allowed and intentionally fall back to a default icon

For service cards:

• the footer was intentionally simplified to icon-only actions for stable first-class links only
• current stable card actions are:
  • repository
  • runbook
  • OpenAPI
  • delivery / CI when a URL exists
• free-form links[] are intentionally excluded from the card footer because icon-only affordances become ambiguous there
• card action selection lives in src/lib/service-card-links.ts

Small helper extraction rule

When a page/component repeats a small piece of non-trivial logic more than once, prefer extracting it into a tiny tested helper instead of keeping parallel inline copies.

Current examples:

• src/lib/catalog/service-summary.ts — shared fallback summary derivation
• src/lib/page.ts — shared route/page response helpers

This is the preferred level of abstraction here:

• small and boring helpers are good
• generic framework-y abstraction layers are not

Recent implementation notes

Mermaid diagram rendering

• Mermaid diagrams are authored as fenced ```mermaid code blocks in the service Markdown body
• Rendered client-side by the full mermaid npm package (v11); all diagram types are supported
• src/scripts/mermaid-render.ts finds <pre><code class="language-mermaid"> blocks produced by markdown-it and replaces them with rendered SVGs
• Each diagram gets a collapsible "Show source" toggle; syntax errors show an error notice with source expanded
• Script is wired via <script src="..."> in ServiceDocumentationCard.astro — Astro bundles it into the client
• securityLevel: 'strict' is set to prevent script injection from diagram sources
• The mermaid bundle adds ~500 KB to the client bundle; that is inherent to the library and acceptable for this use case
• PlantUML and Structurizr remain proposed — they would require either a remote render endpoint (kroki.io) or a local Java process, both worth a separate decision

Managed Git scheduler and sync behavior

• Managed Git sync previously happened on the request path and caused repeated pulls, race conditions, and noisy logs.
• This has been replaced with an in-process scheduler that syncs on startup and then periodically.
• Requests now read from local checkout state and do not perform Git sync work.
• Sync is locked per checkoutPath so overlapping requests or cycles do not clone/pull the same source concurrently.
• The old git pull --ff-only origin <branch> approach caused brittle behavior (Cannot fast-forward to multiple branches).
• Current sync behavior is closer to a disposable cache model:
  • git fetch origin <branch>
  • git checkout <branch>
  • git reset --hard origin/<branch>
• Invalid partial checkout directories are removed before retrying clone.

Logging direction

• Startup/config logs were too noisy when config was recomputed per request.
• Config is now cached so those logs are emitted once per process instead of on every page render.
• Logging now goes through a small shared helper instead of raw console.* calls spread across runtime modules.
• Default log output stays human-readable text for local development.
• LOG_FORMAT=json enables structured one-line logs for Kubernetes-style environments and log aggregation.
• Useful log categories now include:
  • scheduler startup
  • startup sync cycle start/finish
  • interval sync cycle start/finish
  • per-source sync start/success/failure with duration
  • snapshot build/refresh success and stale fallback behavior
  • scan scope and discovered file counts
  • parse/validation warnings with per-file details for both local and Git-backed sources
• The system now keeps a validated in-memory catalog snapshot and serves requests from that snapshot instead of rescanning and reparsing on each page render.
• After sync cycles, servdir refreshes the snapshot in the background and keeps serving the last known good catalog if a refresh fails.
• The cache/snapshot logic now lives as an explicit catalog cache subsystem instead of being hidden inside load.ts, to make later stats, debug views, and observability easier to extend.

Dual deployment modes

• Servdir now supports two deployment flavors:
  • default Node server runtime
  • explicit static export mode
• Static export is additive and should not destabilize the main server path.
• Current static flow is aimed at GitHub Pages first, with base-path-aware internal links and a dedicated Pages workflow.
• Static mode intentionally skips runtime-only concerns such as middleware auth enforcement and scheduler-driven Git sync.
• Static build was re-verified after the catalog status card refactor, and pnpm build:static still completes successfully.
• Build/test reminder for future work:
  • baseline: pnpm test && pnpm build
  • when relevant to build/routing/deployment paths: pnpm build:static

Catalog entry model broadening

• The catalog started service-first, but the model now supports a broader optional kind field.
• If kind is omitted, it defaults to service.
• Current examples of broader entry types include:
  • application
  • tool
• This is meant to broaden the catalog without renaming the whole product or breaking older service definitions.
• Service definitions now also support an optional structured tech_stack object with shared categories:
  • languages
  • frameworks
  • data
  • platform
  • tooling
• Intentional design choice: keep tech_stack structured enough for future icons/grouping/filtering, but do not make it kind-specific yet.

Tag navigation

• Visible tags now link to dedicated tag pages.
• The catalog now has:
  • /tags for the tag index
  • /tags/[tag] for tag-specific listings
• Important implementation lesson: do not nest tag links inside a row-level anchor in list views. That broke the compact list and had to be fixed by restructuring the row markup.

SSH behavior and local container testing

• App-managed Git checkout/pull should prefer SSH repository access keys over provider API tokens.
• Common container/Kubernetes defaults are:
  • /etc/servdir/ssh/id_ed25519
  • /etc/servdir/ssh/known_hosts
• The implementation no longer forces IdentitiesOnly=yes in the default GIT_SSH_COMMAND because that was stricter than many working local SSH setups.
• Important operational lesson: mounting ~/.ssh into a container is not equivalent to having the host SSH environment.
• In the tested local setup, host ssh -Tv git@bitbucket.org succeeded because authentication used ssh-agent, not just the raw private key file.
• Containerized Git without agent forwarding may still prompt for a passphrase even if host git clone works normally.
• For docs and real deployments, the recommended path remains a dedicated repository access key without passphrase.
• For local debugging, mounting personal SSH files can work, but it is an interactive convenience path, not a good production-like example.

Documentation added or improved

• Added docs/kubernetes.md for Kubernetes deployment, Flux/SOPS-friendly config patterns, SSH setup, and operational notes.
• Added docs/service-definition.md describing the service.md contract:
  • required and optional front matter fields
  • Markdown body behavior
  • validation behavior
  • discovery rules for local and managed Git sources
• Discovery now also supports an explicit single-repo entry file at the scan root: .servdir.md.
  • keep using services/*/service.md for multi-entry catalogs
  • use root .servdir.md when the repository itself should appear as one catalog entry
• Added ToCs to README.md, docs/kubernetes.md, and docs/service-definition.md.
• README now has a direct link to the service definition reference, similar to the Kubernetes guide link.

Service detail page layout (sectioned + sticky TOC)

• The detail page (src/pages/services/[id].astro) was reorganized from a two-column "doc + right-rail cards" layout into a single scrollable column of named sections with a sticky right-rail table of contents (task-uwc, direction "A").
• Section order and anchors: overview, documentation, apis, operations, dependencies, quality. Each is a shareable anchor; the TOC highlights the section in view via IntersectionObserver (rootMargin: -20% / -70%), not scroll math.
• Tech stack is deliberately NOT a section: it renders inside the Overview masthead so a service's "what it is" and "what it's built with" sit together at the top. On wide screens (xl+) the masthead is two columns — identity on the left, the tech stack as a bordered "Tech stack" spec card top-right (label + badge rows). Below xl the masthead collapses to a single column and the stack card flows to the bottom. The stack card only appears when there is stack to show, so empty or sparse stacks never leave an awkward empty box. Grouping/relabel logic lives in src/lib/catalog/tech-stack.ts; tech_stack.platform is relabeled "Runtime" to avoid colliding with the free-form platform attribute.
• Masthead identity column ordering: name, then tags directly under the name, then description, the attribute strip, and the repository link. Validation state is intentionally NOT shown in the header — it lives in the Quality section, so the masthead stays about identity, not status.
• src/lib/catalog/service-sections.ts is the single source of truth for presence: the page body, the section components' own empty-state guards, and the TOC all derive from the same predicates, so rendered sections and TOC entries cannot drift.
• Always-on vs conditional decision: overview, dependencies, and quality always render — their empty states ("No dependencies declared", "No validation issues") are meaningful in a catalog, and they were always visible before. documentation, apis, and operations render only when they carry data.
• New surfacing vs before: the page now shows description, the free-form platform string (in the Overview attribute strip, distinct from tech_stack.platform), and the tech_stack.* groups (in the Overview Stack block) — none of which had a home previously.
• Section headings are owned by ServiceSection (h2, text-title); the three retained cards (ServiceDocumentationCard, ServiceDependenciesCard, ServiceValidationCard) had their internal CardTitle removed to avoid double headings. ServiceHeader and ServiceMetadataCard were retired.
• Known follow-up (heading hierarchy): when a service.md body starts with an # H1 (e.g. # Bike API), that h1 now renders directly under the "Documentation" section heading and duplicates the service name shown in the Overview masthead. Markdown rendering was intentionally left untouched in this pass; suppressing/demoting a leading body h1 is a separate decision.
• Inline OpenAPI rendering (a hosted API reference embedded in the "APIs & interfaces" section) is direction "B", tracked as task-z8y. The section currently links out; z8y will embed the viewer and decide spec sourcing (external URL + CORS vs spec-in-repo) in an ADR.

Inline OpenAPI rendering (APIs section)

• openapi[] entries now accept a file (spec stored beside service.md in the catalog repo) in addition to url. See ADR 014.
• Sourcing: the spec is read as a raw string in the service-page model (loadServiceApiSpecs) — at build time in static mode, per request in SSR. No server-side YAML parsing, no cross-origin fetch (sidesteps CORS). The string is baked into the page as island props.
• Renderer: Scalar (@scalar/api-reference-react), lazy-loaded via dynamic import only when a reader expands a spec, so the heavy bundle never ships on spec-less pages and is never imported during SSR.
• Embedding: collapsible inline. Each in-repo spec is a disclosure in the APIs section; external url specs render as a link out; an unreadable file degrades to a plain label.
• Card-footer (service-card-links.ts) still surfaces the OpenAPI icon only for url entries; file-only specs live on the detail page.
• The local catalog now contains real service descriptions generated from public non-fork aholbreich GitHub repositories. Repositories with sparse READMEs are described conservatively and may need human refinement.

Current UI component structure

Reusable UI building blocks currently in use:

• src/components/ui/Badge.astro — compact status/tag pills
• src/components/ui/Card.astro — shared card shell
• src/components/ui/IssueList.astro — validation issue rendering
• src/components/ui/KeyValueList.astro — metadata key/value and link list rendering
• src/components/ui/MetadataPill.astro — compact icon-plus-label chip for service attributes
• src/components/ui/SectionTitle.astro — consistent section headings
• src/components/ui/ServiceCard.astro — dense reusable service list card
• src/components/catalog/ServiceCatalogGrid.astro — catalog index shell: toolbar, view panels, empty-state handling
• src/components/catalog/ServiceCardGrid.astro — card grid (<ul>) with per-item filter data attributes; used in both flat and grouped card view
• src/components/catalog/CatalogHero.astro — index-page hero section for catalog identity and intro text
• src/components/catalog/CatalogStatusCard.astro — compact catalog status summary card, now mostly a composition shell
• src/components/catalog/CatalogStatusTabs.astro — icon-only tab controls for the status card
• src/components/catalog/CatalogStatusPanel.astro — tab panel wrapper with shared accessibility wiring
• src/components/catalog/CatalogStatusDetailList.astro — reusable detail grid for configuration/runtime/issues sections
• src/components/catalog/CatalogStatusGitSourcesPopover.astro — small overlay for managed Git source details
• src/lib/catalog/service-summary.ts — shared service excerpt fallback logic for card/list-style summaries
• src/lib/page.ts — app-aware route helpers such as 404 redirect responses
• src/lib/catalog/service-sections.ts — detail-page section model: ordered descriptors + presence predicates (single source of truth for which sections render and the TOC)
• src/lib/catalog/tech-stack.ts — tech stack grouping/presence for the Overview Stack block (category order + the platform→"Runtime" relabel)
• src/components/catalog/ServiceSection.tsx — shared section shell (anchor id + text-title heading)
• src/components/catalog/ServiceOverviewSection.tsx — detail-page masthead (name, tags-under-name, description, attribute strip, repo, and a top-right tech stack spec card on xl+); replaced the former ServiceHeader
• src/components/catalog/ServiceApisSection.tsx — APIs section: in-repo specs as collapsible inline references, external specs as links
• src/components/catalog/ServiceApiReference.tsx — single-spec collapsible disclosure that lazy-loads the Scalar renderer on expand
• src/lib/catalog/openapi.ts — resolves openapi[] entries to specs (reads in-repo file content as a raw string; path-contained; graceful fallback)
• src/components/catalog/ServiceOperationsSection.tsx — runbook, delivery, and free-form links[]
• src/components/catalog/ServiceTableOfContents.tsx — sticky right-rail scroll-spy nav (IntersectionObserver)
• src/components/catalog/ServiceDocumentationCard.tsx — detail-page documentation body card (Mermaid); title now owned by its section
• src/components/catalog/ServiceDependenciesCard.tsx — detail-page dependency list; title now owned by its section
• src/components/catalog/ServiceValidationCard.tsx — detail-page validation state card; title now owned by its section

Design reminder:

• keep repeated route responses and small presentation derivations in lib helpers once they appear in more than one page/component

• prefer extending these components or adding adjacent catalog-scoped components before pushing more layout logic back into page files

• keep generic UI primitives in src/components/ui/

• keep domain-aware catalog components in src/components/catalog/

• keep page files mostly orchestration plus layout composition, not presentation-heavy mapping

Future direction: architecture diagrams and Pulumi context

• There is user interest in surfacing Pulumi-generated architecture drawings from the service catalog.
• Current recommendation is to show architecture context on the same service detail page, not on a separate page.
• Short-term pragmatic option: represent Pulumi drawings as links.
• Likely future product direction: support a more explicit architecture/diagram field once the shape is clearer.
• If this becomes first-class, likely questions are:
  • remote URLs only vs local files stored beside service.md
  • generic architecture references vs Pulumi-specific modeling
  • how diagram previews should be rendered on the service page

Open questions

• Should provides become a first-class field in the initial schema?
• Should validation status be stored only in memory, or exposed through a small explicit model?
• Should local development support file watch and hot reload for catalog changes in the first implementation?
• What is the smallest useful runtime sync status model to expose, if a health/debug endpoint is added?
• When repository scanning arrives, what is the smallest acceptable persistent state model?
• Should sync metadata eventually use JSON files or SQLite?
• Should repository scanning be feature-flagged at first?

Future feature ideas

Repository scanning

Likely next feature after the MVP foundation:

• configure one or more repositories
• periodically fetch or sync them
• scan for service definition files
• merge or expose findings through the catalog model
• track sync status and last run state

Validation UX

Potential useful additions:

• validation details page
• warnings vs errors distinction
• unresolved dependency view
• duplicate id diagnostics

Architecture diagrams

Potential useful additions:

• support Pulumi-generated architecture diagrams on service detail pages
• start with generic link-based support if needed
• later consider first-class architecture metadata
• evaluate whether diagram assets should be remote only or allowed inside the catalog repo next to service.md

Authoring support

Potential later additions:

• example templates for new services
• schema docs for maintainers
• CLI helper for generating a new service entry

Suggested near-term improvements

1. Expose a small health or debug view for sync state, for example last sync time, last success, last error, and source status.
2. Improve validation UX in the UI, especially duplicate id diagnostics, unresolved dependency visibility, and clearer warning vs error treatment.
3. Decide whether provides should become a first-class field in the service definition schema.
4. Decide how Pulumi and other architecture diagrams should be modeled first: generic links, first-class metadata, or local assets beside service.md.
5. Tighten logging further, potentially with log levels or a quieter default mode for routine scans.
6. Add more targeted tests around managed Git sync behavior, especially failed startup sync, invalid checkout recovery, and multi-source behavior.
7. Decide whether runtime sync status should remain in memory only or get a small explicit model that can be exposed operationally.
8. Consider a cleaner application startup hook for operational subsystems like the scheduler instead of relying on module initialization.
9. Extend the explicit catalog cache subsystem with stats/debug surfaces if operational visibility becomes important.

Authentication mode (2026-05-19)

AUTH_MODE (none | basic | oidc) replaced the boolean basicAuth.enabled config flag. The three modes are documented in ADR 012, docs/authentication.md, and the README "Authentication" section. New env vars on top of the existing BASIC_AUTH_*:

• AUTH_MODE
• AUTH_OIDC_TENANT_ID, AUTH_OIDC_CLIENT_ID, AUTH_OIDC_CLIENT_SECRET, AUTH_OIDC_REDIRECT_URI
• AUTH_SESSION_SECRET (>= 32 bytes after base64 decode)
• AUTH_SESSION_TTL_HOURS (optional, default 8, range 1..168)

Backwards compatibility: if AUTH_MODE is unset but the legacy BASIC_AUTH_ENABLED=true is present, the runtime infers AUTH_MODE=basic with a one-line warning. Existing basic-auth deployments continue to work without a config change.

Operator gotchas to keep in mind:

• Rotating AUTH_SESSION_SECRET invalidates every live session instantly. A re-login wave is expected.
• /health/live and /health/ready bypass auth in every mode; pinned by src/middleware.test.ts.
• Static export mode (SERVDIR_BUILD_MODE=static) is unauthenticated by design — host it behind whatever the static target provides.
• OIDC callback token exchange must use the configured AUTH_OIDC_REDIRECT_URI, not the internal adapter request origin; otherwise reverse-proxied Node deployments can exchange a code with http://localhost:<port>/auth/callback and Entra rejects it with AADSTS500112.

Deploy-time theming (2026-05-19)

ADR 013 introduces a single deploy-time theme file selected via UI_THEME_CONFIG. Default behavior is unchanged when the variable is unset.

Key implementation details to remember:

• src/lib/theme.ts is a separate module, deliberately not part of AppConfig. Theming is a layout concern; mixing it into the domain config would log entire token objects through the config dump.
• BaseLayout.astro sets <html data-theme="custom"> and injects :root[data-theme="custom"] { … } rules. The [data-theme] selector bumps specificity above the baseline :root rules in src/styles/tokens.css, so the override wins regardless of where Astro emits the bundled stylesheet <link>.
• App title stays on CATALOG_TITLE. The theme schema covers logo and favicon only; titles are deployment-level and we kept a single source of truth.
• If a theme provides no dark block, the boot script does not auto- apply dark from system prefs and ThemeToggle returns null.
• Static export bakes the theme at build time. To switch themes on a static deployment, rebuild with a different UI_THEME_CONFIG. This is documented in docs/theming.md and ADR 013.
• Token validation is strict: unknown keys reject the whole file and fall back to default. Adding a new token requires extending TOKEN_CSS_VAR in src/lib/theme.ts.

Working assumption

The catalog is the product. Repository scanning is a future ingestion mechanism, not the foundation of the system.