Skip to content

feat(autofix): Add pr_iteration endpoint step and operator activity#117218

Merged
joseph-sentry merged 15 commits into
masterfrom
joey--autofix-pr-iteration-entrypoints
Jun 16, 2026
Merged

feat(autofix): Add pr_iteration endpoint step and operator activity#117218
joseph-sentry merged 15 commits into
masterfrom
joey--autofix-pr-iteration-entrypoints

Conversation

@joseph-sentry

@joseph-sentry joseph-sentry commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Accept the pr_iteration step in the explorer autofix endpoint, gated behind
the autofix-pr-iteration feature flag. The step requires an existing run with
at least one created pull request, otherwise it returns a 400. Map the
SEER_ITERATION_COMPLETED webhook to a SEER_ITERATION_COMPLETED group activity
in the operator, carrying the pull requests and iteration index.

Also, transform user IDs stored in run state feedback into serialized users for frontend

@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Jun 9, 2026
@joseph-sentry joseph-sentry force-pushed the joey--autofix-pr-iteration-hook branch from 680b16b to 5b2ee97 Compare June 9, 2026 19:04
@joseph-sentry joseph-sentry force-pushed the joey--autofix-pr-iteration-entrypoints branch from 8f21984 to 2dd9635 Compare June 9, 2026 19:04
@joseph-sentry joseph-sentry force-pushed the joey--autofix-pr-iteration-hook branch from 5b2ee97 to ad7c62e Compare June 10, 2026 20:25
@joseph-sentry joseph-sentry force-pushed the joey--autofix-pr-iteration-entrypoints branch from 2dd9635 to c4da8d7 Compare June 10, 2026 20:25
@joseph-sentry joseph-sentry force-pushed the joey--autofix-pr-iteration-hook branch from ad7c62e to 6daae98 Compare June 12, 2026 14:10
@joseph-sentry joseph-sentry force-pushed the joey--autofix-pr-iteration-entrypoints branch 2 times, most recently from 905c091 to c4da8d7 Compare June 12, 2026 15:21
@joseph-sentry joseph-sentry force-pushed the joey--autofix-pr-iteration-hook branch 2 times, most recently from ad7c62e to 91aba65 Compare June 12, 2026 16:31
@joseph-sentry joseph-sentry force-pushed the joey--autofix-pr-iteration-entrypoints branch from c4da8d7 to 39bd905 Compare June 12, 2026 16:31
Comment thread src/sentry/seer/endpoints/group_ai_autofix.py Outdated
@joseph-sentry joseph-sentry force-pushed the joey--autofix-pr-iteration-entrypoints branch from 39bd905 to 84f4687 Compare June 12, 2026 17:06
@joseph-sentry joseph-sentry force-pushed the joey--autofix-pr-iteration-hook branch 2 times, most recently from d11ddf4 to 8cbe651 Compare June 12, 2026 17:40
@joseph-sentry joseph-sentry force-pushed the joey--autofix-pr-iteration-entrypoints branch 2 times, most recently from 5842a6e to 9687a5b Compare June 12, 2026 19:48
@joseph-sentry joseph-sentry force-pushed the joey--autofix-pr-iteration-hook branch from 8cbe651 to 1a608b0 Compare June 12, 2026 19:48
Comment thread src/sentry/seer/endpoints/group_ai_autofix.py
@joseph-sentry joseph-sentry force-pushed the joey--autofix-pr-iteration-hook branch from 11bbfa9 to 03037bc Compare June 15, 2026 14:07
@joseph-sentry joseph-sentry force-pushed the joey--autofix-pr-iteration-entrypoints branch from 2fce8d3 to e8eebdf Compare June 15, 2026 14:07
@joseph-sentry joseph-sentry force-pushed the joey--autofix-pr-iteration-hook branch from 03037bc to c26e4e4 Compare June 15, 2026 14:51
@joseph-sentry joseph-sentry force-pushed the joey--autofix-pr-iteration-entrypoints branch from e8eebdf to 5bc7bf9 Compare June 15, 2026 14:51
Base automatically changed from joey--autofix-pr-iteration-hook to master June 15, 2026 15:22
@linear-code

linear-code Bot commented Jun 15, 2026

Copy link
Copy Markdown

CW-1399

@joseph-sentry joseph-sentry marked this pull request as ready for review June 15, 2026 18:29
@joseph-sentry joseph-sentry requested a review from a team as a code owner June 15, 2026 18:29
Comment thread src/sentry/seer/autofix/autofix_agent.py
Comment thread src/sentry/seer/autofix/on_completion_hook.py
@joseph-sentry joseph-sentry force-pushed the joey--autofix-pr-iteration-entrypoints branch from ae176eb to a824c10 Compare June 15, 2026 18:33

@Zylphrex Zylphrex left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few things but overall LGTM

Does this need to be added to the on completion hook as well so it responds in slack and we fire a webhook? Can be in a follow up, just wondering what the plan is.

message=user_context,
source={
"type": "user-ui",
"user_id": request.user.id,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a thought but why not just store the user here and avoid having to hydrate the user later?

This is manual process so my expectations are that this wont take up that much storage anyways

pull_requests = event_payload.get("pull_requests", [])
if pull_requests:
activity_data["pull_requests"] = pull_requests
elif event_type == SentryAppEventType.SEER_ITERATION_COMPLETED:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain what this activity does?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mostly here for parity with the other steps having activities, but i don't think this ends up doing anything right now

there is code that seems like it'll handle activities in the future, that's behind a feature flag and there's currently no handlers implemented for these activities

there's also an activity log in the UI where we show these, that's behind a feature flag

@joseph-sentry joseph-sentry force-pushed the joey--autofix-pr-iteration-entrypoints branch from a824c10 to 8b754c1 Compare June 15, 2026 19:34
if step == AutofixStep.PR_ITERATION and run_state is not None:
if step == AutofixStep.PR_ITERATION:
if run_state is None or not run_state.repo_pr_states:
raise PrIterationNoPullRequestException()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR iteration allows non-created PRs

Medium Severity

pr_iteration is only blocked when repo_pr_states is empty, not when every entry lacks a created PR. Runs with only creating or error states still pass validation and return 202, which conflicts with the documented requirement for at least one created pull request.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 8b754c1. Configure here.

joseph-sentry and others added 12 commits June 15, 2026 16:24
Send the ITERATION_COMPLETED webhook when a PR_ITERATION step finishes,
including the pull request payload, code changes, and iteration index, and
record the iteration index on the introspection analytics event. Add
introspect_iteration to evaluate whether revised iteration changes are ready
to update the existing pull request, and route PR_ITERATION through it. Extract
the pull-request and code-changes payload builders into shared helpers.

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
Accept the pr_iteration step in the explorer autofix endpoint, gated behind
the autofix-pr-iteration feature flag. The step requires an existing run with
at least one created pull request, otherwise it returns a 400. Map the
SEER_ITERATION_COMPLETED webhook to a SEER_ITERATION_COMPLETED group activity
in the operator, carrying the pull requests and iteration index.

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
_hydrate_feedback_users assumed every block message carried a metadata
dict, but messages can have metadata set to null. Calling .get() on the
result crashed when hydrating feedback users. Coalesce metadata to an
empty dict before reading the feedback field.

Co-Authored-By: Claude <noreply@anthropic.com>
Patch get_autofix_run_state (not get_autofix_agent_state) in pr_iteration
tests, and add the feedback=None kwarg to trigger_autofix_agent call
assertions.
@joseph-sentry joseph-sentry force-pushed the joey--autofix-pr-iteration-entrypoints branch from 8b754c1 to 09b1bb2 Compare June 16, 2026 12:27
Comment thread src/sentry/seer/endpoints/group_ai_autofix.py
Comment on lines 388 to +389
iteration_index: int | None = None
if step == AutofixStep.PR_ITERATION and run_state is not None:
if step == AutofixStep.PR_ITERATION:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The get_iteration_for_insert_index function lacks bounds checking for insert_index and safe access for metadata["iteration_index"], risking unhandled exceptions.
Severity: HIGH

Suggested Fix

Add validation to ensure insert_index is within the bounds of the state.blocks list before access. When retrieving iteration_index from a block's metadata, use the .get() method with a default value or wrap the access in a try...except KeyError block to prevent unhandled exceptions.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: src/sentry/seer/autofix/autofix_agent.py#L388-L389

Potential issue: In the `pr_iteration` flow, the function
`get_iteration_for_insert_index` uses a user-provided `insert_index` to access
`state.blocks[insert_index]` without performing a bounds check. It also accesses
`metadata["iteration_index"]` without a safe fallback like `.get()`. An out-of-bounds
`insert_index` will raise an `IndexError`, and a missing `iteration_index` key will
raise a `KeyError`. Neither of these exceptions are caught by the endpoint's existing
exception handlers, which will cause an unhandled 500 server error on the public API.

Did we get this right? 👍 / 👎 to inform future reviews.

user_id: int
# The serialized user, resolved at write time so the read path doesn't have
# to hydrate it. ``None`` if the user could not be serialized.
user: NotRequired[Any]

@sentry-warden sentry-warden Bot Jun 16, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-serialized user (all emails, options, flags) stored in Seer feedback metadata and returned to other org members

When submitting pr_iteration feedback, the endpoint serializes the requesting user with as_user=serialize_generic_user(request.user). Because as_user equals the target user, UserSerializer._user_is_requester is True and the result is a UserSerializerResponseSelf containing the user's full registered email list, user options, and internal flags. This payload is stored in feedback.source["user"], embedded into the Seer message's prompt_metadata["feedback"], and round-tripped back through block.dict() in the GET endpoint, where any authenticated org member with group-read access can read another member's personal emails, options, and flags. Only a minimal attribution subset (id, name, avatarUrl) is needed.

Evidence
  • group_ai_autofix.py:329-331 calls user_service.serialize_many(filter={"user_ids":[request.user.id]}, as_user=serialize_generic_user(request.user)); since as_user is the same user, UserSerializer._user_is_requester returns True.
  • users/api/serializers/user.py:193-203 then populates emails (all UserEmail rows), options, and flags, producing UserSerializerResponseSelf rather than the public response.
  • The result is stored as feedback.source["user"] (Feedback in autofix_agent.py) and embedded via prompt_metadata["feedback"] = json.dumps({... "source": feedback.source ...}) at autofix_agent.py:418.
  • Message.metadata is dict[str, str] | None (agent/client_models.py:33) and is part of MemoryBlock; the GET handler returns blocks = [block.dict() for block in state.blocks] at group_ai_autofix.py:442, surfacing message.metadata.feedback to any org member with group-read access.
Also found at 2 additional locations
  • src/sentry/seer/endpoints/group_ai_autofix.py:68-69
  • src/sentry/seer/endpoints/group_ai_autofix.py:326-342

Identified by Warden wrdn-data-exfil, wrdn-pii · GE3-KTX

- serialize with users with as_user = None to avoid storing too much information
  in run state feedback
- correct logic and update check of pr_iteration_enabled in group_ai_autofix to check metadata in run
  state, when run state is available: the desired behaviour is to respect the metadata.pr_iteration_enabled
  field, over the value of the feature flag when the endpoint is called

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

There are 3 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit d36c8a5. Configure here.

return Response(
{"detail": "run_id is required for pr_iteration"},
status=status.HTTP_400_BAD_REQUEST,
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing endpoint-level feature flag check for pr_iteration

Medium Severity

The endpoint checks that resolved_run_id is not None for pr_iteration, but never checks features.has("organizations:autofix-pr-iteration", ...) before calling trigger_autofix_agent. The test test_pr_iteration_requires_feature_flag asserts mock_trigger_explorer.assert_not_called() and expects a 403, but since trigger_autofix_agent is mocked (and won't raise PrIterationNotEnabledException), the endpoint proceeds normally and returns 202. An endpoint-level feature flag gate is needed here to match the documented intended behavior.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit d36c8a5. Configure here.

Comment thread tests/sentry/seer/autofix/test_autofix_agent.py
Comment on lines +317 to +321
and user_context is not None
and request.user
and request.user.is_authenticated
):
# Serialize the user here on write so the read path (GET) doesn't have

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The check user_context is not None allows empty strings, creating unnecessary Feedback objects and triggering RPC calls even when no meaningful context is provided.
Severity: LOW

Suggested Fix

Change the condition at line 317 from if user_context is not None to if user_context. This truthy check will correctly filter out empty strings and prevent the creation of meaningless feedback objects.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: src/sentry/seer/endpoints/group_ai_autofix.py#L317-L321

Potential issue: When a client sends an empty string for `user_context`, the condition
`user_context is not None` evaluates to true. This results in the creation and storage
of a `Feedback` object with an empty message. This behavior leads to several
inefficiencies: an unnecessary RPC call to `user_service.serialize_many()`, storage of
meaningless feedback data in the run's metadata, and a misleading `has_user_context`
metric being set to `"yes"`. The intended behavior, as seen in `build_step_prompt`, is
likely to only process feedback when `user_context` is a non-empty string.

@joseph-sentry joseph-sentry merged commit ea7423d into master Jun 16, 2026
65 of 66 checks passed
@joseph-sentry joseph-sentry deleted the joey--autofix-pr-iteration-entrypoints branch June 16, 2026 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants