Curated Knowledge Stores: Source-Mapped, File-Backed, Traceable Knowledge Management

[jflowers]

Vision

A curated knowledge store is a persistent, file-backed knowledge repository that:

1. Maps sources to stores via configuration — "content from these sources should be curated into this knowledge store"
2. Compounds and aggregates across sources — meeting notes + Slack conversations + GitHub issues merge into unified knowledge articles
3. Full traceability — every curated fact traces back to its source document, block, and timestamp
4. File-backed (markdown) — version-controllable, portable, human-readable, committable to git

Architecture Sketch

sources.yaml                    knowledge-stores.yaml
├── disk-meetings               stores:
├── disk-slack-export     ──▶     - name: team-decisions
├── github-issues                   sources: [disk-meetings, disk-slack-export, github-issues]
├── disk-specs                      path: .uf/dewey/knowledge/team-decisions/
└── code-dewey                    - name: architecture
                                    sources: [disk-specs, code-dewey]
                                    path: .uf/dewey/knowledge/architecture/

Curation Pipeline

Source documents → LLM extraction → Knowledge Store (md files)

The LLM reads indexed source content and extracts:

• Decisions with temporal ordering (newer supersedes older)
• Facts and context that carry forward until contradicted
• Patterns that accumulate across sources
• References to external docs/tools

Each extracted piece includes full source attribution.

Knowledge File Format

---
tag: authentication
category: decision
created_at: 2026-04-10T14:30:00Z
sources:
  - id: disk-meetings
    document: sprint-planning-2026-04-10.md
    block: "## Decisions Made"
    excerpt: "Team decided to switch from API keys..."
  - id: disk-slack-export
    document: design-decisions/2026-04-12.md
    block: "9:20 AM — alice"
    excerpt: "OAuth2 timeout increased to 60s..."
confidence: high
supersedes: [authentication-1]
---

Use OAuth2 for authentication (changed from API keys
per sprint planning 2026-04-10). Timeout: 60s (increased
from 30s per user feedback in Slack #design-decisions).
Rate limit: 100 req/min (unchanged from original design).

How It Differs from Current Learnings

Aspect	Current Learnings	Curated Knowledge Stores
Input	Agent calls store_learning manually	Automatic extraction from indexed sources
Source tracing	None — just the learning text	Full: source ID, document, block, excerpt
Aggregation	Single-source per learning	Multi-source — merges across meetings, Slack, issues
Persistence	SQLite only (lost if DB deleted)	Markdown files (git-trackable, portable)
Trigger	Agent decides what to store	Configuration-driven — "curate these sources into this store"
Quality	Varies — depends on agent judgment	LLM-curated with confidence scoring

Open Design Questions

Q1: Unified with learnings or separate?

A. Unified — Knowledge stores replace the current learnings system. store_learning writes to a knowledge store.
B. Separate layers — Learnings are quick agent notes. Knowledge stores are curated, source-traced extractions. Both feed into dewey compile.
C. Knowledge stores subsume learnings — Current store_learning is deprecated.

Q2: When does curation happen?

A. On-demand — dewey curate CLI/MCP tool
B. After indexing — dewey index triggers curation on configured stores
C. Continuous — Background process watches for new content

Q3: Configuration location?

A. Separate file — .uf/dewey/knowledge-stores.yaml
B. In sources.yaml — Each source declares knowledge_store: <name>
C. Hybrid — Store definitions in a separate file, source mapping in sources.yaml

Q4: How do knowledge stores interact with search?

A. Auto-indexed — Knowledge store markdown files are indexed as a disk source (they are just .md files)
B. Special source type — A type: knowledge-store source with custom handling
C. Direct integration — Knowledge store content goes straight into the search index during curation

Q5: Trust tier for curated content?

A. New tier — curated (between authored and draft)
B. Use existing draft — Curated content starts as draft, gets promoted via dewey promote
C. Use existing authored — Curated content is derived from authored sources, so it inherits the tier

Cross-Project Impact

• Dewey — New curation pipeline, knowledge store config, file persistence
• gcal-organizer (feat: export extracted decisions as local markdown for Dewey indexing jflowers/gcal-organizer#12) — Meeting decision export feeds knowledge stores
• get-out (feat: write local markdown copies of exported conversations for Dewey indexing jflowers/get-out#4) — Slack conversation export feeds knowledge stores
• All UF repos — Knowledge stores would be shared via git, giving every repo access to curated team knowledge

Related Issues

• feat: persist learnings as markdown files for backup and portability dewey#50 — Persist learnings as markdown (precursor to knowledge stores)
• feat: index meeting decisions and Slack exports from gcal-organizer and get-out dewey#46 — Index meeting decisions and Slack exports
• feat: export extracted decisions as local markdown for Dewey indexing jflowers/gcal-organizer#12 — Export meeting decisions as markdown
• feat: write local markdown copies of exported conversations for Dewey indexing jflowers/get-out#4 — Export Slack conversations as markdown
• Spec 013 (knowledge compilation) — Temporal intelligence and compilation infrastructure

Discussion Prompt

This is a significant architectural addition. Before speccing it:

1. Does the source-to-store mapping model feel right, or should curation be more ad-hoc?
2. Should curated knowledge stores replace the current learnings system or complement it?
3. What confidence/quality threshold should curated facts meet before being stored?
4. Should knowledge stores be project-scoped (per-repo) or org-scoped (shared across repos)?

[jflowers]

Knowledge Quality Analysis in Curation

During the curation process, the LLM should not only extract facts — it should identify quality issues in the source material. This turns curation into both an extraction AND a quality assessment step.

1. Missing Information

Pattern	Example	Flag
Decision without rationale	"Going with Option B" — but why?	missing_rationale
Action without owner	"Need to migrate the database" — who? when?	missing_owner
Requirement without criteria	"Should be fast" — how fast?	missing_acceptance_criteria
Undefined term reference	"Use the new auth flow" — which one?	undefined_reference

2. Implied Information

Pattern	Example	Flag
Unstated assumption	"Deploy on Tuesday" implies deploy window is active	implied_assumption
Inherited context	"Same approach as last time" — needs resolution	unresolved_reference
Implicit dependency	"After the API is ready" — dependency not tracked	implied_dependency
Default value	"Use standard timeouts" — what is standard?	implicit_default

3. Incongruent Information

Pattern	Example	Flag
Cross-source contradiction	Meeting: 30s timeout / Slack: 60s timeout	contradiction
Stale reference	Spec references a reversed decision	stale_reference
Scope conflict	Issue says "API only" / meeting says "API and CLI"	scope_conflict
Timeline conflict	"Q3" vs "end of April" — same or different?	timeline_conflict

4. Confidence Scoring

Each curated fact carries a confidence level:

Confidence	Criteria
High	Explicitly stated, no contradictions, multiple sources agree
Medium	Explicitly stated but single-source, or minor implied context
Low	Implied/inferred, or contradictions exist across sources
Flagged	Missing critical info or unresolvable contradictions — needs human review

5. How This Appears in Knowledge Files

---
tag: authentication
category: decision
confidence: medium
quality_flags:
  - type: missing_rationale
    detail: "No explanation for why API keys were rejected"
    source: disk-meetings/sprint-planning-2026-04-10.md
  - type: implied_dependency
    detail: "Implies OAuth2 provider is already configured"
  - type: incongruent
    detail: "Timeout differs between meeting (30s) and Slack (60s)"
    resolution: "Used Slack value (60s) as more recent"
    sources:
      - disk-meetings/sprint-planning-2026-04-10.md (30s)
      - disk-slack-export/design/2026-04-12.md (60s)
---

6. Lint Integration

dewey lint would surface knowledge store quality:

• "3 curated facts have confidence: low"
• "5 facts have missing_rationale flags"
• "2 unresolved contradictions across sources"

This gives teams visibility into knowledge quality and where human review is needed.

7. The Curation Prompt

The LLM curation prompt would include instructions like:

For each extracted fact:

1. Assign a confidence level (high/medium/low/flagged)
2. If a decision is stated without reasoning, flag missing_rationale
3. If the fact implies something not explicitly stated, flag the implication
4. If the fact contradicts another source, flag incongruent with both sources and attempt temporal resolution (newer wins)
5. If critical context is missing (who, when, why), flag the specific gap

This transforms curation from "copy the important bits" into "extract, assess, and annotate the important bits with quality metadata."

[jflowers]

Git-Backed Knowledge Stores

Curated markdown files are inherently git-friendly — plain text, YAML frontmatter, deterministic filenames. The question is where the git repo lives and how sync works.

Three Models

Model A: In-Repo
Knowledge lives at .uf/dewey/knowledge/<store>/ inside the project repo. Committed alongside code.

• ✅ Zero infrastructure — team gets knowledge via git pull
• ❌ Knowledge scattered across repos, no single org-wide view

Model B: Dedicated Knowledge Repo
A separate repo (e.g., org/knowledge-base on GitLab) aggregates curated knowledge from all projects. Dewey pushes curated files after curation.

• ✅ Single source of truth for org-wide knowledge
• ❌ Requires push permissions, token management, merge conflict potential

Model C: Hybrid
Knowledge lives in-repo (A) AND syncs to a central repo (B). Local copy is the working copy; central repo is the aggregated view.

• ✅ Best of both — local access + org-wide view
• ❌ Sync complexity, potential for drift

GitLab-Specific Considerations

For self-hosted/enterprise GitLab:

1. GitLab Files API: POST /api/v4/projects/:id/repository/files/:file_path pushes curated files without needing a full git clone. Avoids SSH key management.
2. GitLab CI validation: .gitlab-ci.yml in the knowledge repo runs dewey lint on every push to validate quality — stale decisions, contradictions, confidence scores.
3. Merge Requests as quality gate: Instead of pushing directly, Dewey creates an MR with curated files. Human reviews before knowledge enters the central repo.
4. GitLab wiki alternative: GitLab project wikis are Git-backed. The knowledge store could BE the wiki repo.

Configuration

# .uf/dewey/knowledge-stores.yaml
stores:
  - name: team-decisions
    sources: [disk-meetings, disk-slack-export]
    local_path: .uf/dewey/knowledge/team-decisions/
    
    git_sync:
      enabled: true
      remote: git@gitlab.company.com:org/knowledge-base.git
      branch: main
      path_prefix: team-decisions/
      strategy: merge-request  # push | merge-request
      auth: ssh-key            # ssh-key | token | oauth

Sync Workflow

dewey curate --store team-decisions
  ├── 1. Extract knowledge from sources
  ├── 2. Write markdown files locally
  ├── 3. If git_sync.enabled:
  │      ├── Clone/pull remote (sparse checkout)
  │      ├── Copy curated files to path_prefix/
  │      ├── strategy=push: commit + push
  │      └── strategy=merge-request: create MR via API
  └── 4. Log sync result

Cross-Project Traceability

When knowledge syncs to a central repo, the source traceability must include which PROJECT the knowledge came from:

---
tag: authentication
category: decision
project: unbound-force/dewey
sources:
  - id: disk-meetings
    document: sprint-planning-2026-04-10.md
    project: jflowers/gcal-organizer
  - id: disk-slack-export
    document: design/2026-04-12.md
    project: jflowers/get-out
---

This means a reader in the central knowledge repo can trace any fact back to its originating project, source type, and specific document.

Open Questions

Q6: Which model — A (in-repo), B (dedicated repo), or C (hybrid)?
My lean: C (hybrid) for orgs with multiple projects. A (in-repo) for single-project teams.

Q7: Push vs merge-request for sync?
My lean: merge-request by default (quality gate). Push for trusted/automated pipelines.

Q8: GitLab vs GitHub vs generic git?
The sync layer should be pluggable — git push works everywhere. GitLab/GitHub MR/PR creation is an optional convenience layer on top.

Q9: What happens on merge conflicts in the central repo?
If two projects curate knowledge about the same topic simultaneously, the central repo could have conflicts. Options: last-writer-wins, manual resolution, or separate files per project (no conflicts possible).