Skip to content

internal/source: Provider interface and concurrent collector #9

Description

@Saber5656

Title

internal/source: Provider interface and concurrent collector

Summary

Define the Provider interface (the future plugin boundary) and implement
the collector that runs enabled providers concurrently with per-source
timeouts, gathering events and warnings without letting any single source
fail the run (DESIGN.md §3.1, §6.1).

Context

Four providers (10, 12→via gitsrc provider, 14, 15) implement this
interface; daily/standup/sources commands call the collector. The
"sources degrade, reports never fail" property lives here.

Scope

  • internal/source/provider.go, collector.go, tests with fake providers.

Detailed Requirements

  1. Interface exactly per DESIGN §6.1:

    type Provider interface {
        ID() model.SourceID
        Collect(ctx context.Context, rng timeutil.Range) ([]model.Event, []model.Warning, error)
    }
  2. Collect(ctx, providers []Provider, rng timeutil.Range, opts Options) Result:

    • Options{PerSourceTimeout time.Duration} (default 30s when zero);
      Result{Events []model.Event, Warnings []model.Warning}.
    • Runs each provider in its own goroutine with
      context.WithTimeout(ctx, PerSourceTimeout).
    • Timeout or context cancellation → warning
      model.WarnSourceTimeout (source_timeout) carrying the provider id;
      provider results arriving after timeout are discarded.
    • Provider error → converted to a warning
      (Code = string(id)+"_failed" unless the provider already returned
      typed warnings; the error message becomes the warning message);
      partial events returned alongside an error are kept.
    • Panic in a provider is recovered → warning (<id>_panic), run
      continues (a malformed source file must never crash the CLI —
      invariant I5).
    • Output ordering is deterministic: events sorted with
      model.SortEvents, warnings sorted by (Source, Code, Message) —
      regardless of goroutine completion order.
  3. FilterByIDs(providers []Provider, ids []model.SourceID) []Provider
    supports the --source flag (order-preserving; unknown ids are the CLI's
    problem, already validated in 07).

  4. Providers receive their config at construction (each provider issue owns
    its constructor); the collector knows nothing about config.

  5. Zero providers → empty Result, no warnings (the CLI decides what an
    empty report looks like).

Acceptance Criteria

  • Fake-provider tests: fast+slow (slow exceeds timeout → its events
    dropped, source_timeout warning present, fast source unaffected);
    erroring provider (warning, others unaffected); panicking provider
    (recovered, warning, others unaffected); partial-events-with-error
    kept.
  • Determinism: run the same mixed set 50× and assert byte-identical
    marshaled Result (ordering rule works under scheduling variance).
  • Context cancellation from the caller stops in-flight providers
    (fake provider observes ctx.Done within its loop; asserted).
  • -race clean.
  • Package imports: stdlib + internal/model + internal/timeutil only.

Validation

go test -race -count=20 ./internal/source/ (repeat count shakes out
ordering flakes) pasted into the PR.

Dependencies

03, 04, 05 (types only — providers take typed config structs defined in 05).

Non-goals

Any real source parsing (09–15), sanitize/redact stages (18 owns them),
subprocess plugin execution (v2, ADR/DESIGN non-goal).

Design References

  • docs/DESIGN.md §3.1 (pipeline), §6.1 (interface), §13 (source_timeout)

Source of truth: docs/issues/08-source-provider-collector.md (PR #1, branch docs/v1-design). If this issue and the repo docs disagree, the docs win. Execution order and dependencies: docs/ISSUE_PLAN.md (this is issue 08 of 33).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions