Skip to content

fix(ai): answer every prompt from the engine-issued candidate domain - #6973

Merged
matthewevans merged 2 commits into
mainfrom
ship/ai-answers-from-issued-domain
Aug 3, 2026
Merged

fix(ai): answer every prompt from the engine-issued candidate domain#6973
matthewevans merged 2 commits into
mainfrom
ship/ai-answers-from-issued-domain

Conversation

@matthewevans

@matthewevans matthewevans commented Aug 3, 2026

Copy link
Copy Markdown
Member

choose_action answered a prompt from one of three sources, and only two of
them selected from AiDecisionContract's domain. The seven specialist
heuristics CONSTRUCTED a GameAction off the waiting_for payload and hoped
the contract accepted it. PR #6829 added that guard without a source of truth,
which converted "AI submits an illegal action" into "AI submits nothing" — and
aiController.ts cannot distinguish a refusal from "this seat owes nothing",
so it halts after three with ai-controller-stuck:<prompt>.

Reported shape: an AI cast Praetor's Grasp against an 88-card opponent library
and hung. SearchChoice enumeration was capped at 12 candidates while the
tutor scorer ranked all 88, so the AI's own argmax was unsubmittable.

  • candidates.rs: apply the beam cap only to genuinely combinatorial
    enumerations. C(n,0)+C(n,1) is linear, so a single-card search now issues
    the whole pool. Gifts Ungiven (count=4) is unaffected.
  • search.rs: deterministic_choice's SearchChoice arm ranks the engine's
    issued SelectCards candidates instead of re-deriving a pool; the local
    BEAM_K enumerator and score_search_choice_cards are deleted. All seven
    specialist sites bind to the contract and FALL THROUGH on a miss rather than
    hard-returning None. The OpponentGuess and card-predicate samplers draw
    from the issued actions, preserving CR 608.2d uniformity.
  • ai_support/mod.rs + filter.rs: structurally_valid_search_selection
    lets SimulationFilter skip its clone-and-apply probe for search
    selections. All three conditions the submission guard checks (cardinality,
    pool membership, CR 608.2c selection constraint) are decidable without
    mutating state; scoped searches defer to the simulation. Uncapping alone
    cost 217ms/88 candidates; with this the same list validates in 180us with
    zero legality clones.

Gates: choose_action_never_answers_outside_the_engine_issued_domain mounts
the invariant at choose_action altitude — #6964's gate sits on
fallback_action, which the seven specialists return before, and stayed green
on a tree carrying this P0. Verified RED on a reverted tree.

Summary by CodeRabbit

  • Bug Fixes

    • Improved AI search decisions across large card pools.
    • Prevented invalid, duplicate, or out-of-scope card selections.
    • Ensured AI choices remain within the actions available in each game state.
    • Improved fallback behavior when specialized decisions cannot be used.
    • Improved card selection ranking by prioritizing value and combination benefits.
  • Performance

    • Reduced unnecessary search combinations while preserving complete evaluation for simple searches.
  • Tests

    • Added coverage for broad search pools, selection validity, action availability, and fallback decisions.

`choose_action` answered a prompt from one of three sources, and only two of
them selected from `AiDecisionContract`'s domain. The seven specialist
heuristics CONSTRUCTED a `GameAction` off the `waiting_for` payload and hoped
the contract accepted it. PR #6829 added that guard without a source of truth,
which converted "AI submits an illegal action" into "AI submits nothing" — and
`aiController.ts` cannot distinguish a refusal from "this seat owes nothing",
so it halts after three with `ai-controller-stuck:<prompt>`.

Reported shape: an AI cast Praetor's Grasp against an 88-card opponent library
and hung. `SearchChoice` enumeration was capped at 12 candidates while the
tutor scorer ranked all 88, so the AI's own argmax was unsubmittable.

- `candidates.rs`: apply the beam cap only to genuinely combinatorial
  enumerations. C(n,0)+C(n,1) is linear, so a single-card search now issues
  the whole pool. Gifts Ungiven (count=4) is unaffected.
- `search.rs`: `deterministic_choice`'s `SearchChoice` arm ranks the engine's
  issued `SelectCards` candidates instead of re-deriving a pool; the local
  `BEAM_K` enumerator and `score_search_choice_cards` are deleted. All seven
  specialist sites bind to the contract and FALL THROUGH on a miss rather than
  hard-returning `None`. The `OpponentGuess` and card-predicate samplers draw
  from the issued actions, preserving CR 608.2d uniformity.
- `ai_support/mod.rs` + `filter.rs`: `structurally_valid_search_selection`
  lets `SimulationFilter` skip its clone-and-apply probe for search
  selections. All three conditions the submission guard checks (cardinality,
  pool membership, CR 608.2c selection constraint) are decidable without
  mutating state; scoped searches defer to the simulation. Uncapping alone
  cost 217ms/88 candidates; with this the same list validates in 180us with
  zero legality clones.

Gates: `choose_action_never_answers_outside_the_engine_issued_domain` mounts
the invariant at `choose_action` altitude — #6964's gate sits on
`fallback_action`, which the seven specialists return before, and stayed green
on a tree carrying this P0. Verified RED on a reverted tree.
@matthewevans
matthewevans enabled auto-merge August 3, 2026 20:30
@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: cc9023cd-22bd-40f8-9dc2-20564e4bd4cf

📥 Commits

Reviewing files that changed from the base of the PR and between c37fbd4 and 9d5469a.

📒 Files selected for processing (1)
  • crates/phase-ai/src/policies/tutor.rs

📝 Walkthrough

Walkthrough

The engine preserves full pools for linear search actions and validates non-scoped selections structurally. The phase AI pipeline binds specialist, fallback, and scored decisions to engine-issued actions and scores complete issued selections.

Changes

AI action contract alignment

Layer / File(s) Summary
Engine search candidate validity
crates/engine/src/ai_support/candidates.rs
Zero- and one-card searches use the full card pool. Tests cover enumeration and selection structure.
Structural search validation
crates/engine/src/ai_support/mod.rs, crates/engine/src/ai_support/filter.rs
Non-scoped SelectCards actions are checked for cardinality, pool membership, uniqueness, and search constraints before simulation.
Issued-domain AI routing and scoring
crates/phase-ai/src/search.rs, crates/phase-ai/src/policies/tutor.rs
Specialist and fallback decisions validate against issued actions. Search scoring evaluates complete issued selections. Tutor scoring ranks selections by value, combo bonuses, duplicate-name discounts, and value rank.
Issued-domain routing validation
crates/phase-ai/src/search.rs, crates/phase-ai/src/policies/tutor.rs
Tests cover wide pools, contract membership, specialist routing, predicate guesses, non-owing seats, and order-independent selection scoring.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

  • phase-rs/phase#6741: Related changes validate AI search and fallback actions in crates/phase-ai/src/search.rs.
  • phase-rs/phase#6829: Extends engine-issued AiDecisionContract validation in the same AI-support and search paths.
  • phase-rs/phase#6964: Aligns AI search and fallback selections with the engine-issued action domain.

Suggested labels: needs-maintainer

Sequence Diagram(s)

sequenceDiagram
  participant choose_action
  participant AiDecisionContract
  participant SpecialistPolicy
  participant SearchSelectionScorer
  choose_action->>AiDecisionContract: materialize issued actions
  choose_action->>SpecialistPolicy: request specialist action
  SpecialistPolicy->>AiDecisionContract: validate proposed action
  AiDecisionContract-->>choose_action: accept or reject action
  choose_action->>SearchSelectionScorer: score issued SelectCards selections
  SearchSelectionScorer-->>choose_action: return selected issued action
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: ensuring AI answers prompts from the engine-issued candidate domain.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ship/ai-answers-from-issued-domain

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
crates/phase-ai/src/policies/tutor.rs (1)

86-115: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Apply positional decay by card value, not selection order.

search.rs sorts complete selections, not the cards inside each selection. The engine preserves input order when generating combinations, so 0.88^index depends on enumerator order. Compute per-card scores, sort them by descending value, then apply decay by rank to discount the weakest cards.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/phase-ai/src/policies/tutor.rs` around lines 86 - 115, Update
score_search_choice_selection to compute each selected card’s base score first,
sort those scores in descending order, then apply the 0.88 positional decay by
sorted rank so the weakest cards receive the largest discount. Preserve the
existing combo bonus and duplicate-name multiplier before sorting, and sum the
decayed scores without relying on chosen iteration order.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/phase-ai/src/policies/tutor.rs`:
- Around line 86-115: Update score_search_choice_selection to compute each
selected card’s base score first, sort those scores in descending order, then
apply the 0.88 positional decay by sorted rank so the weakest cards receive the
largest discount. Preserve the existing combo bonus and duplicate-name
multiplier before sorting, and sum the decayed scores without relying on chosen
iteration order.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 8dd8c0e6-5a9d-44d9-b466-4c66f231a79d

📥 Commits

Reviewing files that changed from the base of the PR and between ec19b52 and c37fbd4.

📒 Files selected for processing (5)
  • crates/engine/src/ai_support/candidates.rs
  • crates/engine/src/ai_support/filter.rs
  • crates/engine/src/ai_support/mod.rs
  • crates/phase-ai/src/policies/tutor.rs
  • crates/phase-ai/src/search.rs

@matthewevans
matthewevans added this pull request to the merge queue Aug 3, 2026
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown

Generated for head 9d5469a1dff88f522f5b87133a06c91167250a0b.

Parse changes introduced by this PR

✓ No card-parse changes detected.

@matthewevans
matthewevans removed this pull request from the merge queue due to a manual request Aug 3, 2026
`score_search_choice_selection` applied both the redundancy discount and the
`0.88^n` decay by position in the `chosen` slice. That was only ever correct
because the deleted AI-local beam fed it combinations built from a
score-sorted `beam_ids`, so slice position WAS value rank.

Ranking the engine's issued selections removed that precondition without
replacing it: the enumerator emits combinations in pool order and the
candidate list is then sorted by `GameAction::cmp_stable`, i.e. by `ObjectId`.
The decay therefore discounted by object id, and the same two cards scored
0.9472 or 0.9328 depending purely on which id sorted first.

Sort by score before applying either discount, so the score is a function of
the selected SET — which is what "an opponent takes the worst card of the set"
(Gifts Ungiven, CR 608.2c) actually models. The redundancy discount now falls
on the less valuable copy, and the decay on whatever is weakest after it.

Caught by CodeRabbit on #6973.
@matthewevans
matthewevans enabled auto-merge August 3, 2026 21:04
@matthewevans

Copy link
Copy Markdown
Member Author

CodeRabbit's tutor.rs nitpick was confirmed and upgraded — it is a regression this PR introduced, not a pre-existing style point, so it is fixed in 9d5469a rather than deferred.

The positional decay was correct before this PR only by accident of the caller: the deleted AI-local beam built its combinations from a score-sorted beam_ids, so slice position was value rank. Ranking the engine's issued selections dropped that precondition without replacing it — the enumerator emits combinations in pool order and the candidate list is then sorted by GameAction::cmp_stable (i.e. by ObjectId), so the decay discounted by object id.

Measured on the queued commit: the same two cards scored 0.9472 or 0.9328 depending purely on which id sorted first, and the strong card took the discount whenever it sorted second.

Fix sorts by score before applying either discount, making the score a function of the selected set. search_choice_prefers_strongest_single_target now pins both halves — order-independence, and that the decay lands on the weaker card — and was verified red against the previous implementation.

@matthewevans
matthewevans added this pull request to the merge queue Aug 3, 2026
Merged via the queue into main with commit 93da0ca Aug 3, 2026
17 of 19 checks passed
@matthewevans
matthewevans deleted the ship/ai-answers-from-issued-domain branch August 3, 2026 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant