Skip to content

benchmark: expand the public corpus with one new repository slice #34

Description

@vishal2612200

Goal

Add one new public repository slice to the benchmark corpus without overfitting rankings to that repository.

Suggested files

  • benchmarks/public-repos.toml
  • benchmarks/results/YYYY-MM-DD-public.md after running the suite
  • docs/benchmark-learnings.md if the slice changes tuning decisions

Acceptance criteria

  • Adds real historical tasks with expected files.
  • Runs the public benchmark command and records the dated result.
  • Documents any ranking changes as generalizable rules, not repo-specific hacks.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    benchmarkBenchmark cases, metrics, and evidenceenhancementNew feature or requesthelp wantedExtra attention is needed

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions