diff --git a/docs/advanced_memory_strategy.md b/docs/advanced_memory_strategy.md index 53c497d..681584d 100644 --- a/docs/advanced_memory_strategy.md +++ b/docs/advanced_memory_strategy.md @@ -168,22 +168,30 @@ The test: *if this knowledge were lost tomorrow, would we reconstruct it and wri Before writing a new entry, search the target wing and room for semantically similar content. -- If an existing entry covers **80% or more** of the same information: **enrich** the existing entry rather than creating a new one. +- If an existing entry covers the same information (see similarity guidance below): **enrich** the existing entry rather than creating a new one. - If the new entry supersedes an older one: mark the old entry as `STATUS: obsolete` or delete it. - Never create two drawers with the same core rule in different phrasings. +- When enriching an existing entry, preserve its original rule and extend it — do not rewrite it from scratch. Use `mempalace_check_duplicate` as a secondary guard when available. -#### Optional: semantic deduplication thresholds +#### Optional: semantic similarity thresholds -Similarity scoring (e.g. via embeddings) can provide an additional signal when manual judgment is difficult. If your tooling supports it, two thresholds are useful: +For teams who want a more structured signal when deciding whether to enrich or create, similarity scores can serve as **guidance** (not enforcement). Scoped comparison — checking only within the target wing and room — keeps the signal relevant. -- **Soft threshold (~0.5–0.7):** content is related but distinct → review the existing entry and consider enriching it rather than creating a new one. -- **Hard threshold (~0.8–0.9):** content is a near-duplicate → merge into the existing entry or discard the new one. +Suggested ranges as a refinement on top of disciplined writing: -**Scope comparisons to the same wing and room.** Global similarity search produces false positives (unrelated entries that share vocabulary) and false negatives (related entries in different scopes that should not be merged). Scoping to wing + room preserves semantic meaning and keeps the signal useful. +| Similarity | Signal | Suggested action | +|-----------|--------|-----------------| +| ≥ 0.86 | Near-duplicate | Enrich the existing entry; do not create a new one | +| 0.55 – 0.85 | Related content | Review manually; create only if the new entry captures a genuinely distinct rule, contract, or anti-pattern | +| < 0.55 | Likely distinct | Creating a new entry is acceptable if persistence criteria are met | -These thresholds are optional. The primary deduplication strategy remains write discipline: one rule per entry, search before writing, enrich instead of duplicating. +**Type-aware exception:** if two entries are similar in content but differ in type (e.g., one is an `architecture-rule` and the other is an `anti-pattern`), do not merge automatically. Different types serve different purposes and should remain separate even when their similarity score is high. + +These thresholds are an advanced refinement — not a replacement for the writing discipline described above. The primary deduplication mechanism is still human judgment. + +> See [docs/deduplication.md](deduplication.md) for a compact reference on this pattern. ### 5.4 Freshness Metadata diff --git a/docs/deduplication.md b/docs/deduplication.md new file mode 100644 index 0000000..9616ca0 --- /dev/null +++ b/docs/deduplication.md @@ -0,0 +1,63 @@ +# Semantic Deduplication Reference + +> This page is a compact reference. For full context, see [Advanced Memory Strategy](advanced_memory_strategy.md). + +--- + +## Why it matters + +Storing the same rule twice — in different phrasings, different rooms, or different wings — makes retrieval noisy and maintenance expensive. Agents surface contradictory entries. Updates get applied inconsistently. The store degrades. + +Deduplication prevents that. The primary mechanism is **disciplined writing**: before creating a new entry, check whether one already exists. + +--- + +## Scoped comparison + +Always compare a candidate entry only against entries in the **same wing and room**. + +Cross-wing or cross-room comparison produces misleading signals. A rule that is 85% similar to an entry in `myapp/architecture` may still be appropriate in `myapp/anti-patterns` — because the type is different and the purpose differs. + +--- + +## Threshold guidance + +Similarity scores are guidance, not enforcement. Use them to calibrate judgment, not to replace it. + +| Similarity | Signal | Suggested action | +|-----------|--------|-----------------| +| ≥ 0.86 | Near-duplicate | Enrich the existing entry; do not create a new one | +| 0.55 – 0.85 | Related content | Review manually; create only if the new entry captures a genuinely distinct rule, contract, or anti-pattern | +| < 0.55 | Likely distinct | Creating a new entry is acceptable if persistence criteria are met | + +These ranges work as an advanced refinement layered on top of writing discipline. They are not a gate. + +--- + +## Enrich vs. create + +Prefer enriching over creating: + +- **Enrich** when the new information extends, qualifies, or adds rationale to an existing entry. +- **Create** when the new entry captures a genuinely distinct rule, contract, or anti-pattern. +- When enriching, **preserve the original rule** and extend it. Do not rewrite it from scratch. + +--- + +## Type-aware exception + +If two entries are similar in content but differ in type — for example, an `architecture-rule` and an `anti-pattern` that describe the same component — do **not** merge them automatically. Different types communicate different weight and serve different retrieval purposes. Keep them as separate entries. + +--- + +## Human-first rule + +Semantic similarity is a signal. Human judgment is the decision. + +A high similarity score is a prompt to review — not an automatic veto on creating a new entry. If a new entry captures something genuinely distinct, create it regardless of score. If it doesn't, enrich what exists. + +Writing discipline remains the primary deduplication mechanism. + +--- + +*Related: [Advanced Memory Strategy §5.3](advanced_memory_strategy.md#53-deduplication-policy) · [Memory Example](memory_example.md#optional-semantic-deduplication)* diff --git a/docs/memory_example.md b/docs/memory_example.md index ab2060f..6103e59 100644 --- a/docs/memory_example.md +++ b/docs/memory_example.md @@ -92,4 +92,25 @@ This ensures project constraints are applied first, while still benefiting from --- +## Optional: semantic deduplication + +Before adding a new entry to a wing and room, check whether similar content already exists **in that same wing and room**. Scoping the comparison keeps it meaningful. + +**Example using `myapp/architecture`:** + +Suppose you want to add: +> "All external API calls must be wrapped in the repository layer." + +Before creating it, compare against existing `myapp/architecture` entries. + +- If a similar entry already exists (e.g., "Data fetching must go through `src/repositories/`"), **enrich that entry** rather than creating a parallel phrasing. +- If the existing entry covers the same constraint but is phrased differently, extend it and preserve its original rule — don't rewrite it. +- If the types differ — for example, an existing `architecture-rule` vs. a new `anti-pattern` about the same area — keep them as separate entries even if the content overlaps. They serve different purposes. + +Similarity is guidance, not a gate. The primary check is always: *does this entry add something genuinely distinct?* If yes, create it. If not, enrich what exists. + +> For threshold ranges and a compact reference, see [deduplication.md](deduplication.md). + +--- + *For the full strategy, see [Advanced Memory Strategy](advanced_memory_strategy.md).* diff --git a/ros2-architecture-context.instructions.md b/ros2-architecture-context.instructions.md new file mode 100644 index 0000000..eac8bea --- /dev/null +++ b/ros2-architecture-context.instructions.md @@ -0,0 +1,121 @@ +# ROS 2 Architecture Context + +> **Scope:** This instruction file governs how AI agents gather context and persist knowledge +> when working in a ROS 2 project that uses MemPalace as a governed knowledge layer. +> It is a concrete instance of the pattern described in +> [Advanced Memory Strategy](docs/advanced_memory_strategy.md). + +--- + +## Context Gathering + +Before modifying any ROS 2 component, query memory in this order: + +1. Query the **project wing** (e.g., `lifecore_ros2`) for project-specific architecture rules, + component contracts, and anti-patterns. +2. Query the **shared wing** (e.g., `ros2`) for general ROS 2 knowledge — lifecycle semantics, + communication patterns, standard conventions. +3. Merge results. Project entries override shared entries **only when explicitly marked as a + local override**. +4. If MemPalace is unavailable, fall back to `docs/architecture.md`, then `README.md`, + then workspace search. Memory unavailability must never block a task. + +--- + +## Retrieval Priority + +``` +[project wing] → [shared wing] → [local docs] → [code search] + ↓ + merged context + ↓ + decision + ↓ + persist (if durable) +``` + +Conflicts: if a project entry and a shared entry conflict, the project entry wins — but only +when it is explicitly documented as a local override. An undocumented contradiction is a +potential inconsistency, not a silent override. + +--- + +## Persistence + +### What to persist + +Store entries that are **durable**, **repeatable**, and **non-obvious**: + +- Architecture decisions with rationale +- Inter-component contracts +- Confirmed anti-patterns (especially those that caused bugs) +- Conventions that differ from ROS 2 defaults +- Rules that prevent known regressions + +Do **not** persist: debugging steps, temporary session results, facts already in docstrings, +one-time fixes, or speculative ideas not yet validated. + +### Before writing a new entry + +Search the **target wing and room** for semantically similar content before creating +anything new. Scoping the comparison to the same wing and room keeps the signal relevant. + +**Threshold guidance** (guidance, not enforcement): + +| Similarity | Signal | Action | +|-----------|--------|--------| +| ≥ 0.86 | Near-duplicate | Enrich the existing entry; do not create a new one | +| 0.55 – 0.85 | Related content | Review manually; create only if the new entry is genuinely distinct | +| < 0.55 | Likely distinct | Creating a new entry is acceptable if persistence criteria are met | + +**Type-aware exception:** if two entries are similar in content but differ in type +(e.g., `architecture-rule` vs. `anti-pattern`), keep them as separate entries. +Do not merge automatically — different types serve different purposes. + +**When enriching:** preserve the original rule. Extend or qualify it rather than rewriting +it from scratch. The original constraint must remain legible in the updated entry. + +Use `mempalace_check_duplicate` as a secondary guard when available. + +### Entry format + +``` +[type: architecture-rule | component-contract | anti-pattern | code-convention | migration-note] +STATUS: active +CREATED: YYYY-MM-DD + + + +Rationale: + +[see: /] — (optional) +``` + +### Scope assignment + +- **Project wing** (e.g., `lifecore_ros2`): architecture decisions, component contracts, + local conventions, project anti-patterns. +- **Shared wing** (e.g., `ros2`): general ROS 2 knowledge reusable across projects — + lifecycle semantics, standard message patterns, tooling conventions. + +If knowledge is reusable across projects, it belongs in the shared wing, not the project wing. +Do not copy shared content into project entries; cross-reference it instead: + +``` +[see: ros2/lifecycle] — this component extends the state machine described there +``` + +--- + +## Guardrails + +1. Do not write to memory without reading first. +2. Keep entries atomic — one entry, one rule. +3. Tag types explicitly when ambiguous. +4. Mark superseded entries as `STATUS: obsolete` or remove them. +5. Treat obsolescence as maintenance, not optional cleanup. + +--- + +*Reference: [Advanced Memory Strategy](docs/advanced_memory_strategy.md) · +[Deduplication](docs/deduplication.md) · [Memory Example](docs/memory_example.md)*