Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),

## [Unreleased]

### Added
- **User search preferences (`/preferences`, `~/.superhuman/preferences.md`).** `/repo-finder`'s filters were constants baked into the agent prompt (`language:java`, `language:python`), and no command accepted a preference — ask for Go, get Python. They are now sourced from a file the **user** owns. `## Filters` (`languages`, `topics`, `stars`) is mechanical: parsed by `scripts/lib/preferences.sh`, compiled 1:1 into GitHub search qualifiers by `scripts/repo-finder/build_queries.sh`. `## Notes` is prose the agent applies at tie-breaks and issue selection only — never to a numeric score, and disclosed in the shortlist's `notes` field when it does. `topic[:min_stars]` gives one topic its own floor; `any` is reserved and compiles to a query with no `topic:` qualifier; `stars` is a floor and there is no ceiling (a range is a loud error, not a quiet cap). `/repo-finder` gains `--lang` / `--topic` / `--min-stars` overrides that shape one run and never persist. No preferences file means `DEFAULT_PROFILE`, byte-identical to the queries the agent carried inline, so `/contribute` on a fresh machine cannot regress. Covered by `tests/scripts/test_parse_preferences.sh` and `test_build_queries.sh` (network-free, bash 3.2-clean).

### Changed
- **Query shape follows the qualifier asymmetry.** Repeated `language:` unions; repeated `topic:` **intersects**. So languages collapse into one query and each topic gets its own — query count is `|topics|`, and adding a language costs zero queries. Emitting two `topic:` qualifiers on one line silently returns only repos tagged with *every* topic, often none, with exit 0 and no error; `build_queries.sh` refuses to emit one. Same class of silent failure: a typo'd qualifier key and a parenthesized `OR` both return `total_count: 0` with exit 0, so both are rejected at emit time rather than trusted.

### Removed
- **`repo-finder`'s Category Preference Order and its `3e` category bonus (10%).** With filtering server-side, every candidate matches the user's filters by construction, so a bonus for matching is a constant added to every row and ranks nothing. Its weight went to signals that discriminate: responsiveness 30 → **35%**, outside-contributor track 20 → **25%**.

## [0.6.4] — 2026-07-13

## [0.6.3] — 2026-07-12
Expand Down
6 changes: 4 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,10 @@ Thanks for your interest. This is a [Claude Code](https://claude.com/claude-code
superhuman/
├── .claude-plugin/ # plugin.json + marketplace.json manifests
├── agents/ # subagent prompts (the behavioral logic) + SHARED_STATE.md
├── commands/ # slash commands (/contribute, /repo-finder, …)
├── commands/ # slash commands (/contribute, /repo-finder, /preferences, …)
├── scripts/ # shell extracted from agent prompts, organised by agent
│ ├── lib/ # shared helpers: state.sh, mistakes.sh, flake.sh, delim.sh
│ ├── lib/ # shared helpers: state.sh, preferences.sh, mistakes.sh, flake.sh, delim.sh
│ ├── repo-finder/ # build_queries.sh — compiles preferences.md into search queries
│ ├── profiler/ scorer/ orchestrator/ builder/
├── schemas/ # JSON Schema (draft 2020-12) for every shared-state file
├── tests/scripts/ # bash unit tests for scripts/ (one per script/schema)
Expand All @@ -29,6 +30,7 @@ Behavior is split deliberately:

- **Agent prompts (`agents/*.md`)** hold the reasoning, decisions, and safety prose. Things a model needs to *judge* stay here.
- **Scripts (`scripts/*.sh`)** hold the deterministic shell — JSON reads/writes, state transitions, gates. Things a computer should *execute the same way every time* live here.
- **User config (`~/.superhuman/preferences.md`)** holds what the *user* wants. Everything else under `~/.superhuman/` is machine-written state the plugin owns; this one file is theirs, and only `/preferences` writes it on their behalf.

When you find yourself writing more than a couple of lines of `bash`/`jq` inside an agent prompt, extract it to `scripts/` and call it from the prompt. That keeps prompts short (cheaper to load per phase) and makes the logic testable.

Expand Down
59 changes: 57 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,9 +176,52 @@ Codex does not have slash commands. Use these prompts to invoke the same workflo
| `/contribution-fleet [N]` | Not supported in Codex — fleet runs require parallel subagent dispatch. Run sequential loops instead. |
| `/contribution-dashboard [owner/repo]` | `Use the superhuman skill to show the contribution dashboard for owner/repo (or all repos if omitted).` |
| `/repo-finder [N]` | `Use the superhuman skill to refresh my open-source repo shortlist with up to N candidates.` |
| `/preferences` | `Use the superhuman skill to set my repo search preferences.` |
| `/preferences --show` | `Use the superhuman skill to show my repo search preferences and the queries they compile to.` |

## Usage

**Tell it what you want to work on** — run `/preferences` once. Without it, the scan uses a default profile (AI/ML, Java, Python, dev-tools, >20K stars), which is fine but is not *yours*:

```text
/preferences
```

Comment thread
coderabbitai[bot] marked this conversation as resolved.
It asks four things — languages, topics, a star floor, and anything else in your own words — then writes `~/.superhuman/preferences.md`:

```markdown
## Filters
languages: go, rust
topics: backend, infrastructure, llm:5000, any
stars: 2000

## Notes
Prefer small focused libraries over sprawling frameworks.
I'd rather fix bugs than add features — feature work invites bikeshedding.
Nothing that needs a GPU to run the test suite.
```

The two halves do different jobs, and the split is deliberate:

- **`## Filters` is mechanical.** It compiles 1:1 into GitHub search qualifiers. "Only Go and Rust" means only Go and Rust. `topic[:min_stars]` gives one topic its own floor (`llm:5000`); `any` is reserved and means "…plus repos carrying none of these tags"; `stars` is a **floor**, and there is no ceiling.
- **`## Notes` is prose.** Nothing can hard-filter on "focused rather than sprawling", so the agent applies your notes where judgment actually happens — breaking ties between repos, and choosing which issue to fix — and it tells you in the shortlist's `notes` field when your prose drove the pick. It never touches a score.

You can edit the file by hand; it is yours. `/preferences --show` prints it back along with the exact queries it compiles to, so a filter is never opaque:

```text
Your next /repo-finder will search:
q='language:go language:rust topic:backend stars:>2000 archived:false'
q='language:go language:rust topic:infrastructure stars:>2000 archived:false'
q='language:go language:rust topic:llm stars:>5000 archived:false'
q='language:go language:rust stars:>2000 archived:false'
```

One query per topic — languages are free (they union inside a single query, so adding one costs nothing). To deviate for a single run without touching the saved file:

```text
/repo-finder 5 --lang rust --topic cli
```

**Single run** — let the orchestrator pick everything:

```
Expand Down Expand Up @@ -231,7 +274,8 @@ Orchestrator + 10 specialists + 1 shared-state contract document.
| Command | Purpose |
|---|---|
| `/contribute [owner/repo] [issue#]` | One full end-to-end contribution. Loopable. |
| `/repo-finder [N]` | Refresh `repo-shortlist.json` with up to N candidate repos (default 10, max 25). |
| `/preferences` | Set what `/repo-finder` searches for — languages, topics, star floor, free-form notes. Writes `~/.superhuman/preferences.md`. `/preferences --show` prints the current file and the queries it compiles to. |
| `/repo-finder [N] [--lang X] [--topic Y] [--min-stars Z]` | Refresh `repo-shortlist.json` with up to N candidate repos (default 10, max 25). The flags override your saved preferences **for that run only** — they never write to disk. |
| `/contribute-loop [N]` | Run N sequential contributions (default 3, max 20). Stops on `suspicious_halt` or `crash`. |
| `/contribution-fleet [N \| owner/repo ...]` | Launch N parallel contributor runs. |
| `/contribution-dashboard [owner/repo]` | Read-only view of active run, score history, plateaued dimensions, iteration cap, recent merge outcomes, latest loop. |
Expand Down Expand Up @@ -265,9 +309,11 @@ superhuman/
│ ├── contribute-loop.md
│ ├── contribution-dashboard.md
│ ├── contribution-fleet.md
│ ├── preferences.md
│ └── repo-finder.md
├── scripts/ # Versioned shell extracted from agent prompts (v0.5.0+)
│ ├── lib/ # Shared helpers: state.sh, mistakes.sh, flake.sh, delim.sh, lesson_checks.sh
│ ├── lib/ # Shared helpers: state.sh, preferences.sh, mistakes.sh, flake.sh, delim.sh, lesson_checks.sh
│ ├── repo-finder/ # build_queries.sh — compiles preferences.md into GitHub search queries
│ ├── profiler/ # repo-profiler steps + scan_structure/write_repo_scan/dossier_fresh
│ ├── scorer/ # merge-probability-scorer step implementations
│ ├── orchestrator/ # opensource-contributor + reputation_gate.sh + write_run_summary.sh
Expand All @@ -286,6 +332,7 @@ All persistent state lives under `~/.superhuman/`. Per-repo state is keyed by `<

```
~/.superhuman/
├── preferences.md # YOURS. Written by /preferences, read by repo-finder.
├── repos/
│ └── apache-airflow/
│ ├── repo_profile.json
Expand Down Expand Up @@ -317,6 +364,8 @@ All persistent state lives under `~/.superhuman/`. Per-repo state is keyed by `<

File ownership (sole-writer + readers) is documented in `agents/SHARED_STATE.md`. Every shared-state file has a matching JSON Schema (draft 2020-12) under `schemas/`, validated at write time.

`preferences.md` is the exception, and its location says so: `global/` holds cross-repo *state* (mostly agent-written, with user-editable exceptions like `repo_blocklist.json`), while `preferences.md` sits at the root because it is not state at all — it is **input**. No agent writes it; `/preferences` writes it on your behalf.

## Safety rails

- **Impact audits before refactors.** `builder` invokes `impact-auditor` before applying any reviewer-suggested refactor to a shared function. Blocks the class of bug where "just read `self.app.config` instead of calling `conf.get()`" is correct at Flask request time and fatal at FastAPI startup.
Expand All @@ -340,6 +389,12 @@ See [SECURITY.md](./SECURITY.md) for the full safety model and how to report a v
**What is superhuman?**
superhuman is an autonomous open-source contribution agent for [Claude Code](https://claude.com/claude-code) and Codex. It's a multi-agent harness that picks an open issue in a repository, writes the fix, scores its own pull request on a 10-dimension merge-probability rubric, and iterates on the weakest dimension until a maintainer would merge it.

**How do I make it look for the kind of repos I actually care about?**
Run `/preferences`. It asks for languages, topics, a star floor, and anything else in your own words, then writes `~/.superhuman/preferences.md` — a file you own and can hand-edit. Filters are hard (only Go and Rust means only Go and Rust); the notes are advisory and steer which issue gets picked. `/preferences --show` prints the exact GitHub queries your answers compile to, so you can see the filter rather than trust it. Without a preferences file, the scan uses a sensible default profile.

**Why does `/repo-finder` return fewer repos than I asked for?**
Because your filters matched that many. It reports honestly ("12 of 63 candidates matched; showing 12") and will not widen a query to pad the list — filler is what made the shortlist feel random in the first place. Loosen the filters (drop a topic, lower `stars`, add `any`) if you want more.

**How is it different from Copilot, Cursor, Devin, or a plain coding agent?**
Most AI coding tools are *open-loop*: they generate a change once and hope it's good. superhuman is *closed-loop*. It grades its own pull request before opening it, spends each iteration fixing the single weakest dimension, re-scores, and stops only when it converges — gradient descent on one objective: *will a maintainer merge this?*

Expand Down
20 changes: 20 additions & 0 deletions agents/SHARED_STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,26 @@ Repo-agnostic: `~/.superhuman/global/`
| `lessons_global.jsonl` | lesson-distiller | planner, builder, merge-probability-scorer |
| `lesson_regressions.jsonl` | lesson-distiller | merge-probability-scorer, future `/contribution-dashboard` |

User-owned: `~/.superhuman/`

| File | Owner | Readers |
|------|-------|---------|
| `preferences.md` | **user** (form: `/preferences`) | repo-finder |

`preferences.md` sits at the root of `~/.superhuman/`, not under `global/`, and
the location is the point: `global/` is **cross-repo state**, mostly written by
agents (`repo-shortlist.json`, `repo_cooldown.json`, `merge_outcomes.jsonl`) with
a couple of user-edited exceptions (`repo_blocklist.json`). `preferences.md` is
not state at all — it is *input*, the only file here that exists purely to tell
the plugin what you want. No agent writes it; `/preferences` writes it on your
behalf, and repo-finder only reads it.

Its `## Filters` block is mechanical (parsed by `scripts/lib/preferences.sh`,
compiled to search qualifiers by `scripts/repo-finder/build_queries.sh`). Its
`## Notes` block is prose the agent applies at tie-breaks and issue selection
only — never to a numeric score, and always disclosed in the shortlist's `notes`
field when it did.

`<owner-repo>` is formed as `<owner>-<repo>` (single hyphen; slash replaced).
Example: `apache/airflow` → `apache-airflow`.

Expand Down
16 changes: 12 additions & 4 deletions agents/builder.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,11 @@ edit, check whether its path is present in `entries`.
```bash
GEN="$STATE_DIR/generated_files.json"
if [ -f "$GEN" ]; then
for f in $PLAN_EDIT_FILES; do
# One path per line. `for f in $PLAN_EDIT_FILES` would need the shell to split an
# unquoted expansion; zsh does not, so every path glues into one string, matches
# no entry, and this guard silently passes.
while IFS= read -r f; do
[ -n "$f" ] || continue
MATCH=$(jq --arg p "$f" '.entries[] | select(.path == $p)' "$GEN")
if [ -n "$MATCH" ]; then
RC=$(jq -r '.regenerate_cmd // "null"' <<<"$MATCH")
Expand All @@ -95,7 +99,9 @@ EOF
git -C "$WORKDIR" add "$f"
fi
fi
done
done <<PLAN_FILES
$PLAN_EDIT_FILES
PLAN_FILES
fi
```

Expand Down Expand Up @@ -198,9 +204,11 @@ LANG=$(jq -r '.language // ""' "$STATE_DIR/repo_profile.json" 2>/dev/null)
# Changed-files list = the plan's touch list, one path per line.
CF_TMP="$STATE_DIR/.build_changed.$$"
: > "$CF_TMP"
for f in $PLAN_EDIT_FILES; do
while IFS= read -r f; do
[ -n "$f" ] && printf '%s\n' "$f" >> "$CF_TMP"
done
done <<PLAN_FILES
$PLAN_EDIT_FILES
PLAN_FILES

LESSONS_JSON=$("${CLAUDE_PLUGIN_ROOT}/scripts/lessons/select_lessons.sh" \
--repo "$OWNER_REPO" ${LANG:+--lang "$LANG"} --changed-files "$CF_TMP" \
Expand Down
15 changes: 12 additions & 3 deletions agents/issue-selector.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,15 +66,24 @@ strict `{docs, documentation, doc}` set missed namespaced labels like

```bash
# For each label name:
NORM=$(echo "$LABEL" | tr '[:upper:]' '[:lower:]' | tr '/_:-' ' ')
for tok in $NORM; do
# Split to one token per LINE and read with `while read`, not `for tok in $NORM`:
# zsh does not split unquoted expansions, so the whole label would arrive as one
# token and every docs label would pass the filter.
#
# `-` is NOT a delimiter: it is a literal in the `release-notes` and `copy-edit`
# arms below, and splitting on it makes those two arms unreachable.
NORM=$(echo "$LABEL" | tr '[:upper:]' '[:lower:]' | tr '/_: ' '\n\n\n\n')
while IFS= read -r tok; do
[ -n "$tok" ] || continue
case "$tok" in
*doc*|typo|readme|changelog|releasenote|release-notes|\
grammar|spelling|wording|copy-edit|l10n|i18n|translation|\
website|site|blog|tutorial|example|examples)
skip "docs-label (broad): $LABEL"; break ;;
esac
done
done <<EOF
$NORM
EOF
```

**Filter B — docs-only title pattern (broadened).** Skip (reason
Expand Down
Loading
Loading