Skip to content

feat: co-change + change-entropy git signals in scan (blast-radius input) #297

Description

@rpatricksmith

Summary

Scan git intelligence is currently basic: highChurnFiles = flat per-file commit counts over a 14–30 day window (engine/detectors/git.ts:386-389). Add two richer, deterministic git-derived signals computed in one capped git log --name-only walk:

  1. Co-change pairs — files that historically change together (decay-weighted), so AnaThink can warn "you're touching auth.ts — it usually co-changes with session.ts (not in your scope)."
  2. Change-entropy (Hassan HCM) — files repeatedly caught in chaotic, wide commits, a better hotspot signal than raw churn.

Both feed AnaThink's blast-radius scoping with mechanical signal instead of judgment.

Why it matters

  • Blast-radius scoping — "missing co-change partner" is a concrete "you'll regret this PR" signal we don't have today.
  • Better hotspots — change-entropy captures "this file is repeatedly in messy wide commits," independent of raw churn.
  • Both are git-only and language-agnostic → work for every customer stack.

Current state (researched)

  • Scan git intelligence lives in engine/detectors/git.ts (its own gitExec wrapper, :55-61not utils/git-operations.ts, which is the command-layer wrapper used by pr/work/artifact). Part B work happens against git.ts.
  • The relevant walk already exists: detectRecentActivity (git.ts:359-413) runs git log --since=... --name-only --format="" (:370) and tallies flat counts into highChurnFiles (:386-389). Output shape: GitInfo.recentActivity.highChurnFiles: Array<{path, commits}> (git.ts:49), surfaced in scan.ts:287-292.
  • A dormant typed slot already exists: EngineResult.gitIntelligence (engineResult.ts:277-301) declares churnHotspots, busFactor, coChangeCoupling ({fileA, fileB, coChangePercentage, hasImportRelationship}), bugMagnetFiles — but it is always null (scan-engine.ts:1137) and never populated. We can populate it instead of inventing a new shape.
  • No hard commit cap today — bounded by time window (windowDays 30, narrows to 14 if >300 commits, git.ts:364-367); shallow clones return null (:361-362). Co-change needs broader history, so use a separate capped walk.

Recommended approach

Add one git log --name-only --format='%H' -n <CAP> walk (cap ≈ 1000–2000 commits) in git.ts, parse once into Array<{hash, ageIndex, files[]}>, and compute both signals from that single pass:

  • Co-change: for each commit, for each unordered file pair, accumulate decay-weighted score. Use commit-index decay 0.5^(i/halflife), not timestamp decay — so two scans of the same HEAD are byte-identical regardless of when they run. Skip commits touching > MAX_FILES_PER_COMMIT (≈50) to avoid O(n²) explosion from initial-import / lockfile commits. Emit top-K pairs by score with a support-count threshold (≥2).
  • Change-entropy (HCM): bucket commits into periods; per period compute Shannon entropy H = -Σ pᵢ log₂ pᵢ over the file-change distribution; attribute each file's contribution. High score = repeatedly in wide/chaotic commits.

Both are pure reductions over the same array — deterministic.

Integration points

  • git.ts:359-413 — add a sibling detectChangeCoupling(cwd) / detectChangeEntropy(cwd) called from detectGitInfo (:481-496), leaving the churn-window logic untouched.
  • Reuse git.ts:351-353 SOURCE_EXTENSIONS and isNonProductFilePath (git.ts:9) for filtering, as detectRecentActivity does (:377-382).
  • detectGitInfo (:481-496) — add new fields to the returned object; add matching null entries to the no-repo/no-commit returns (:426-441, :445-460).
  • GitInfo interface (git.ts:11-53) — add coChange / changeEntropy fields.
  • Recommended: also populate the existing EngineResult.gitIntelligence.coChangeCoupling slot (engineResult.ts:289) + add changeEntropyHotspots, wired in scan-engine.ts:1137 (compute in git.ts, assemble in scan-engine). Set hasImportRelationship: false for now — it requires the import graph (separate issue) and is the natural v2 join.
  • Update EngineResult default literals (engineResult.ts:390 git default, :410/:1137 gitIntelligence).

Schema additions

// on GitInfo (git.ts)
coChange: Array<{ fileA: string; fileB: string; score: number; supportCount: number }> | null;
changeEntropy: Array<{ path: string; entropyScore: number; commitCount: number }> | null;

Phasing

  • v1: single capped walk; co-change top-K (commit-index decay, support ≥2, drop mega-commits); per-file change-entropy; expose on GitInfo; render top co-change pair in scan.ts intel section.
  • v2: populate gitIntelligence group; add hasImportRelationship by joining co-change against the import graph (depends on the dependency-graph issue); tune half-life/bucketing; feed AnaThink blast-radius prose.

Gotchas / constraints

  • Determinism: commit-index decay (not wall-clock); sort outputs by (score desc, path asc) for stable tie-breaking.
  • Performance / mega-commits: cap commits AND files-per-commit. Without the per-commit cap, an initial import or lockfile commit dominates.
  • Monorepo: filter via isNonProductFilePath + SOURCE_EXTENSIONS so vendored/fixture/template paths don't pollute pairs.
  • Renames: --name-only doesn't follow renames; document that renames split history (consistent with current highChurnFiles).
  • Shallow clone / no history: mirror detectRecentActivity's null returns — never throw.
  • Test count must not decrease: add tests in the tests/engine/detectors/git-activity.test.ts style (real tmp repo via execSync); existing git tests stay green; new fields default null for no-repo cases.

Acceptance criteria

  1. One new capped git log --name-only walk computes both signals — no second history walk added.
  2. Co-change output is deterministic across repeated scans of the same HEAD (verified by a two-run identical-output test).
  3. Mega-commits (> threshold files) are excluded from co-change (verified by a test where a wide commit creates no spurious top pairs).
  4. Change-entropy ranks a file repeatedly in wide commits above a file with equal raw churn but isolated commits (fixture test).
  5. Shallow/non-git/zero-commit repos return null for the new fields without throwing.
  6. New fields appear in scan.json; (v2) gitIntelligence.coChangeCoupling is populated instead of null; top co-change pair rendered in scan.ts.

Effort

Medium — ~3–4 engineer-days. Walk + two reducers ≈ 150–200 LOC in one file with a clear template (detectRecentActivity). Most time is determinism (decay model, tie-breaking) and the real-tmp-repo fixture tests. v2's import-graph join adds ~1 day. The dormant gitIntelligence type slot removes most schema-design effort.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestfeature-requestNew capabilitypipelinePipeline agents, artifacts, proof chain

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions