Skip to content

Persistent Findings Database and Security Posture Trend Reporting (chainproof history) #64

Description

@Nanle-code

Problem Statement

ChainProof can diff findings between two commits (see the closed "structured diff engine" work), but every scan is otherwise stateless — nothing is retained between runs. Teams adopting ChainProof in CI have no way to answer "is our security posture improving over time," "which findings have been open the longest," or track trends across releases, without manually archiving reports. This is a reasonable expectation for a tool positioning itself as a continuous audit companion, as ChainProof's own README describes it.

Proposed Solution

Add a chainproof history command family backed by a local SQLite database (default .chainproof/history.db, git-ignored by default) that durably records every scan:

  • Every chainproof scan run (opt-in via --record-history, or always-on once .chainproof/history.db already exists) persists: commit SHA, branch, timestamp, and the full finding set (id, rule, file, line, severity, first-seen commit).
  • chainproof history summary [--since <date>] [--branch <name>] — reports finding counts by severity over time, and highlights newly introduced vs. resolved findings between the two most recent recorded scans.
  • chainproof history trend --format json|markdown — emits a time series suitable for plotting (date → severity counts) so teams can render it in their own dashboards or paste it into a markdown report.
  • chainproof history age — lists currently open findings sorted by how many recorded scans (or how much wall-clock time) they've persisted, surfacing long-lived unresolved risk.
  • A finding is matched across scans using the same stable identity logic already used by the diff engine (rule + normalized file + line-range overlap), so a finding that moves a few lines due to unrelated edits is still tracked as the same finding rather than counted as both new and resolved.

Technical Scope

  • New packages/core/src/history/store.ts: SQLite schema and an insert/query layer.
  • New packages/core/src/history/trends.ts: aggregation logic for the summary/trend/age views, reusing the existing finding-identity matching from diff.ts.
  • New packages/cli/src/commands/history.ts wiring the summary, trend, and age subcommands into the existing CLI.
  • Database file path configurable via .chainproofrc.json; writes must be safe for concurrent CI runs (e.g., a busy-timeout/retry) without corrupting existing history.
  • Tests: identity matching across a simulated 5-scan history with findings introduced, resolved, and reintroduced; trend aggregation correctness; concurrent-write safety.

Acceptance Criteria

  • Running five successive scans against a fixture repo with deliberately evolving contracts produces a history summary that correctly reports introduced/resolved counts at each step.
  • history age correctly ranks findings by first-seen date across the recorded history.
  • History recording adds no more than a small, documented constant overhead to scan time.
  • Documentation includes a recommended .gitignore entry and a CI wiring example (e.g., persisting .chainproof/history.db as a CI cache/artifact between runs).

Estimated Scope

Approximately 700 lines of new TypeScript across the store, trend engine, CLI commands, and tests.


Maintainer Scope Upgrade

This issue is being expanded into a substantial, production-quality ChainProof enhancement. The implementation should be designed as maintainable platform work, not as a narrow proof of concept. A successful pull request must provide a cohesive user-facing capability, typed internal APIs, robust tests, documentation, and CI-safe behavior.

Expanded Objective

Add persistent findings history so teams can measure security posture over time rather than treating each scan as stateless. The feature should fit the existing monorepo architecture, reuse current scanner/report/CLI patterns where appropriate, and avoid introducing ad hoc subsystems that are difficult to test or maintain.

Required Implementation Depth

This issue is intentionally scoped to require more than 700 lines of meaningful implementation work. The line count expectation applies to purposeful source, tests, fixtures, and documentation that are necessary to deliver the feature. It must not be satisfied through generated output, lockfile churn, formatting-only changes, duplicated boilerplate, or artificial padding.

Expected work includes:

  • A durable SQLite-backed store with schema migrations, concurrent-write safety, configurable paths, and stable finding identity.
  • chainproof history CLI commands for summary, trend, age, and introduced/resolved reporting.
  • End-to-end tests over evolving fixture repositories plus documentation for CI cache/artifact usage.
  • Public or internal types/interfaces where they clarify behavior and reduce future integration risk.
  • Failure-mode handling for invalid input, missing configuration, unavailable optional dependencies, and degraded execution paths.
  • Documentation updates that explain how maintainers and users should operate the new capability in local and CI environments.

Professional Quality Bar

The implementation must be production-ready and reviewable in isolation. Contributors should include clear separation between parsing, analysis, reporting, CLI/action integration, and persistence or provider code where those concerns apply. The code should follow existing ChainProof conventions, keep behavior deterministic in tests, and avoid coupling core analysis to network-only services unless explicitly optional and mocked.

Acceptance Criteria

  • The delivered PR contains more than 700 meaningful lines of implementation across source, tests, fixtures, and docs, excluding generated files and lockfile-only changes.
  • The feature is integrated into the relevant package entrypoints, CLI commands, report formats, GitHub Action behavior, or documentation as appropriate for this issue.
  • Unit tests cover normal operation, edge cases, invalid inputs, and at least one realistic fixture or end-to-end workflow.
  • Any optional external service, model provider, database, or platform integration has deterministic mocks or fallbacks so CI does not depend on secrets or network availability.
  • User-facing output is documented and stable enough for downstream automation.
  • Backward compatibility is preserved unless the PR explicitly documents a migration path and the maintainer approves it.

Mandatory CI and Merge Requirements

A PR resolving this issue must not be merged until all repository CI checks pass. At minimum, reviewers should verify the following from a clean checkout:

  • npm ci
  • npm run lint
  • npm run build --workspaces --if-present
  • npm run test:ci --workspace=packages/core
  • npm test --workspaces --if-present
  • npm run build --workspace=packages/core && npm run docs --workspace=packages/core
  • Any package-specific tests, examples, validators, or integration checks introduced by the PR

If the PR adds a GitHub Action, report format, dashboard, persistence layer, or external integration, it must also include CI coverage or a documented local verification command for that path. Known warnings are acceptable only when they are documented and do not hide failures.

Review Expectations

Reviewers should reject PRs that only stub APIs, add superficial wrappers, omit tests for critical behavior, rely on live secrets in CI, or meet the line-count target through non-functional bulk changes. The preferred solution is a focused but complete vertical slice that leaves ChainProof more reliable, easier to operate, and easier to extend.

Metadata

Metadata

Assignees

No one assigned

    Labels

    GrantFox OSSIssue tracked in GrantFox OSSMaybe RewardedIssue may be eligible for a GrantFox rewardThird CampaignCampaign: Third CampaignenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions