Skip to content

Latest commit

 

History

History
108 lines (87 loc) · 10.7 KB

File metadata and controls

108 lines (87 loc) · 10.7 KB

Knowledge Base Schema

What This Is

A compounding personal knowledge base focused on general topics, abstract ideas, processes, and subject matters (e.g., "building HVAC control", "token minimization", "knowledge graph generation"). While authors and sources should be recorded (e.g., in frontmatter or a "Sources" section), the primary organization and [[wiki-links]] should focus on concepts, not individuals.

Setup on a New Machine

Linux / macOS (also Git Bash or WSL on Windows):

git clone git@github.com:jlbgit/PersonalKnowledgeBaseCreator.git
cd PersonalKnowledgeBaseCreator
./setup.sh cursor                # Cursor, global scope (default)
./setup.sh cursor global         # explicit global — same as above
./setup.sh claude global         # for Claude Code
./setup.sh copilot global        # for VS Code / GitHub Copilot
./setup.sh cursor claude global  # multiple platforms at once
# From another project directory (optional):
# /path/to/PersonalKnowledgeBaseCreator/setup.sh cursor local

Windows (PowerShell):

git clone git@github.com:jlbgit/PersonalKnowledgeBaseCreator.git
cd PersonalKnowledgeBaseCreator
.\setup.ps1 cursor
.\setup.ps1 cursor global
.\setup.ps1 cursor, claude global

This symlinks the three wiki skills (compile-wiki, ask-wiki, lint-wiki) from skills/ into your platform's skill directory. global (default) also writes wiki-config.md next to those skills (absolute paths to this clone) and creates raw/ and output/ here. local scaffolds a wiki in the current working directory instead — see the README. No external dependencies required. On Windows, symlinks require Developer Mode; the script falls back to copying files if unavailable. The wiki/ folder is plain Markdown — open it in any editor; an optional Obsidian vault config is included if you want graph view and backlinks.

To uninstall: ./setup.sh --uninstall cursor or .\setup.ps1 -Uninstall cursor

How It's Organized

  • raw/: Contains unprocessed source material (Markdown, PDFs, TXT, images). Never modify these files manually. Not tracked in git — stays local.
  • wiki/: Contains the organized wiki and topic pages. AI maintains this entirely. Plain Markdown files; organized into thematic subfolders (e.g., wiki/Agentic AI/, wiki/Building Energy/). An optional .obsidian/ config is included for graph view and backlinks, but no specific viewer is required.
  • output/: Contains generated reports, answers, analyses, and schemas generated by the AI upon request. Not tracked in git — stays local.
  • skills/: Contains the three wiki skill definitions (source of truth). Symlinked into the platform skill directory by setup.sh.
  • Path resolution: Each skill reads wiki-config.md first from the workspace root (project-local wiki), then from the platform skills directory (global wiki). Those files list absolute paths for Wiki root, Wiki folder, Raw folder, and Output folder so the same skills work from any open project.

Wiki Rules

Frontmatter (YAML)

Every topic .md file MUST have valid YAML frontmatter including:

  • title: — Required. The original title of the primary source (paper title, article headline, repo name, etc.). For synthesized concept pages without a single primary source, use the wiki page title. Do not wrap in [[ ]].
  • type: — Required. The kind of source. Must be one of: paper, preprint, article, blog-post, github-repo, book, video, documentation, concept (for synthesized pages without one primary source).
  • authors: — Required. Plain text (list or string). Do not wrap authors in [[ ]].
  • tags: — Required. At least one tag in lowercase-dash format (e.g., agentic-ai, rag).
  • date_added: — Required. ISO 8601 date (YYYY-MM-DD) when this entry was added to the wiki (not the source publication date).
  • url: — Optional. Primary source URL for quick access and Obsidian Dataview (webpage, article, arXiv, DOI link). Omit for synthesized concept pages without one main source.
  • methods: — Optional. List of technologies, tools, frameworks, or experimental/lab methods named in the source (e.g., Neo4j, FastAPI, MCP). Omit when not applicable.
  • description: — Optional. A single-sentence summary of the page. OKF-recommended field (see "Open Knowledge Format (OKF)" below); useful for Dataview and OKF consumers. Omit if the # Title + first paragraph already make the page self-describing.

Page structure

  • Every extracted topic gets its own .md file in wiki/ (thematic subfolders encouraged).
  • After the # Title heading, write an extended summary: 2–4 paragraphs (~250–300 words, roughly up to half an A4 page) covering what the topic is, why it matters, and how it works at a high level.
  • Add a ## Key Takeaways section immediately after the summary: up to 10 bullets with concrete concepts, results, or takeaways (not vague repeats of the summary).
  • Add further sections as needed (e.g. Ecosystem, Challenges) with [[wiki-links]] to related topics.
  • End with a ## Sources section listing citations. Each entry MUST include at least one clickable markdown link to the primary URL (web, DOI, arXiv, etc.), e.g. *Title* by Author — [arXiv](https://arxiv.org/abs/...) | [DOI](https://doi.org/...).
  • Linking philosophy — quality over quantity: Use Obsidian's [[topic-name]] format to build a navigable knowledge graph, but prioritize link quality over quantity. A well-linked page typically has 2–5 outgoing links targeting the most conceptually significant relationships. Link when someone reading page A would genuinely benefit from navigating to page B to deepen their understanding of A's content. Cross-domain links are valuable when the connection is a core contribution (e.g., a paper whose main novelty is applying knowledge graphs to building control should link to both domains). Avoid linking to broad umbrella concepts (e.g., "LLM Agents", "Machine Learning") unless the page is specifically about that concept's mechanism or a direct novel application — hub nodes that everything links to destroy graph structure and make graph-based retrieval inefficient. The text inside [[ ]] must match the target page's filename (without .md); before writing a link, ensure the target page exists in wiki/ or will be created in the same compile run. If a related concept deserves linking but has no page yet, create a brief stub page rather than leaving the link dangling or omitting it.
  • Maintain an index.md in wiki/ that lists every topic with a one-line description (Do NOT use [[wiki-links]] in the index to prevent it from becoming a massive, cluttered hub node in Obsidian).
  • Maintain a log.md in wiki/ to track which raw files have been processed and when.
  • When new raw sources are added, synthesize and update the relevant wiki articles without destroying existing knowledge.

Open Knowledge Format (OKF)

This wiki is compatible with Google's Open Knowledge Format (OKF) — a portable markdown-plus-YAML standard (SPEC.md v0.1, reference implementation) that lets external tooling (e.g. Google's Knowledge Catalog and its HTML graph visualizer) consume a knowledge bundle.

Native conformance. OKF's single required field is a non-empty type: on every non-reserved page — which this wiki already enforces on all pages (see lint_graph.js), with reserved index.md/log.md present. wiki/index.md declares okf_version: "0.1". Our frontmatter maps directly onto OKF's recommended fields:

This wiki OKF field Notes
title title as-is
type type required by both; our enum values are valid OKF type strings
tags tags as-is
url resource URI of the underlying source
date_added timestamp ISO 8601
description description optional single-sentence summary

The one gap: links. We link concepts with Obsidian [[wiki-links]], which OKF consumers do not parse (OKF reads standard markdown / bundle-relative links). The native vault keeps [[wiki-links]] (for Obsidian + agentic graph retrieval); a portable OKF view is produced non-destructively on demand.

Generate the portable bundle: node export_okf.js writes a derived OKF bundle to output/okf/ — it rewrites [[wiki-links]] into bundle-relative markdown links, adds the OKF resource/timestamp aliases, and linkifies the index. The native wiki/ is never modified, and Obsidian never opens output/okf/.

My Interests / Focus Areas

  • (Your Topic Area 1 — e.g., Machine Learning and LLM Architectures)
  • (Your Topic Area 2 — e.g., Software Architecture and System Design)
  • (Your Topic Area 3 — e.g., Your Domain / Industry)
  • (Your Topic Area 4 — e.g., Productivity and Knowledge Management)

⚙️ How to Use This Knowledge Base

1. Ingesting New Files (Daily/Weekly)

Whenever you drop new .md, .pdf, or text files into the raw/ folder: 👉 Run: /compile-wiki

  • What it does: The agent checks wiki/log.md, reads only the unprocessed files, extracts their key concepts, metadata (title, type, authors, methods, URLs), creates/updates topic pages in wiki/ with extended summaries and key takeaways, builds [[wiki-links]] across related topics, and updates the index and log.

2. Querying the Knowledge Base (Ongoing)

When you want to extract insights, find gaps, or generate a report based on your compiled notes: 👉 Ask your question and say: Run /ask-wiki

  • What it does: The agent reads your wiki/ folder to answer your question, saves the full generated report as a Markdown file in the output/ folder, and re-integrates any novel synthesized insights back into the relevant wiki/ pages. This is how the knowledge "compounds" and gets smarter every time you ask a question.

3. Graph Maintenance (Monthly)

After running /compile-wiki a few times, or if your wiki graph starts getting messy/redundant: 👉 Run: /lint-wiki

  • What it does: The agent runs lint_graph.js to validate frontmatter and graph structure, then scans the entire wiki/ directory (ignoring raw/) to clean up the graph. It merges duplicate topics (e.g., "Neural Networks" and "Neural Nets"), fixes broken links, spots contradictions, and suggests 3 new topics to fill gaps in your knowledge graph.