fix(arxiv): over-pick the interests batch, and widen the band it picks from

[Jammy2211]

Both follow-ups from the first live run of arxiv_interests.yml (run 33105368017), which measured what the design had only assumed. Closes the two prompts filed for them:

• draft/bug/pyautomind/over_pick_the_interests_batch_so_the.md
• draft/research/pyautomind/decide_whether_the_interests_ranker_still_earns.md

1. The dedup cost a slot

Claude picked 10; 2608.26039 ("Accretion Disk Sizes … in Lensed Quasars") was already on the strong-lensing inbox from the morning digest, so interests_actions.py append correctly skipped it and the day filed nine. The paper genuinely belongs to the lensing list — the dedup is right and stays.

The fix is slack. PICK_COUNT is now BATCH_SIZE + OVERPICK (13), the prompt returns them ranked, and append trims to its own cap in order after deduping. A day with no overlap still files ten; a day with two overlaps now files ten instead of eight.

The cap is deliberately not restated on this side — interests_actions.py owns it (its BATCH_SIZE, the default of append --limit). Duplicating that number across two repos is how they drift, so the workflow passes no --limit and says why.

2. The ranker had nothing to rank

band_count=61, scored_count=52, truncated=false. The four astro-ph categories announce ~60 papers a day, not the several hundred CANDIDATE_CAP = 60 was sized for. The cap never bound and the keyword net dropped 9 of 61 — the shortlist stage was decoration, and the paging was guarding a volume that doesn't exist.

Rather than retire the stage, spend the headroom on coverage:

Added	Why
gr-qc	black-hole theory, GW, primordial black holes — a real gap; none of it reaches astro-ph
astro-ph.SR	the stellar populations galaxy-evolution work is built on
stat.ME, stat.ML	the inference methods behind the Stats bucket

Eight categories, a few hundred papers a day, and the cap binds again.

Widening alone would have made the list worse

In stat.ME/stat.ML, "Bayesian", "posterior" and "inference" are the house vocabulary — every paper there scores on the Stats terms. A Bayesian method for clinical trials would out-score a lens-modelling paper on keywords alone and crowd it off a 60-paper shortlist.

So HOME_BONUS: a paper whose primary category is astro-ph.* or gr-qc starts +4 ahead. Three properties, each pinned by a selftest check:

• it is a tie-breaker among papers that already matched, never a way in for one that matched nothing (the bonus rides on a non-zero keyword score);
• it is not a filter — a strong stats paper still makes the list, and Claude still judges everything that survives;
• keyword_score and home ride in the candidates JSON, so the prompt can see the thumb on the scale rather than inferring it.

One thing recorded rather than hidden

The filed prompt for (2) said explicitly not to decide on one day's data, and to collect a week including a Monday 3-day band first. That caution was mine, and it was overruled deliberately. Noting it because the widening is the reversible half: if the band turns out unmanageable, CATEGORIES is one tuple.

Testing

arxiv_interests.py --selftest and arxiv_fetch.py --selftest both PASS; pytest tests/ 257 passed. Six new selftest checks: the home/borrowed split, the tie-break between equal-scoring astro and stats papers, that the bonus cannot admit an off-topic paper, that the borrowed paper is kept rather than filtered, that keyword_score is reported beside the boosted score, and that the over-pick arithmetic holds.

Not verified live: arXiv is unreachable from this session's network policy, so the widened query is syntax-checked and the scoring is tested offline only. The next run's announced=/scored=/shortlisted= line is the number to read — it should jump from ~61 into the low hundreds, and shortlisted should now sit at the 60 cap rather than below it.

---

Generated by Claude Code