Skip to content

fix(search-events): stop silent full-text downgrades - #1252

Merged
dcramer merged 8 commits into
mainfrom
fix/search-events-reject-semantic-downgrade
Aug 16, 2026
Merged

dcramer merged 8 commits into
mainfrom
fix/search-events-reject-semantic-downgrade

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Aug 15, 2026 •

Copy link
Copy Markdown
Contributor

Clean up search_events agent orchestration and block the silent false-success path.

Failure mode

Structured filter gets rewritten to full-text:

logs + conv_id:ZYGC-86ZR → message:"*ZYGC-86ZR*"

That looked correct by luck when the id also appeared in log text.

Architecture cleanup

Before: agent rewrite prompt → external validate → different repair prompt → retry up to 3x.

After:

  • one shared agent prompt
  • validateSearch tool inside the agent loop
  • handler keeps a single final validation gate
  • no external multi-attempt repair loop

Guard

Still reject structured → message / log.body downgrades. Keep original filter so validation fails honestly. Real attribute renames still work.

Tests

pnpm --filter @sentry/mcp-core exec vitest run src/tools/catalog/search-events.test.ts src/tools/support/search-events/utils.test.ts (87 passed).

Requested by David Cramer.

--

View Junior Session [Sentry]

sentry-junior Bot and others added 2 commits August 15, 2026 00:02
Stop accepting agent rewrites that turn structured field:value filters into
message/log.body matches. Keep the original filter so validation fails
honestly instead of returning lucky/wrong results. Still allow real attribute
renames and non-semantic repairs.

Co-Authored-By: David Cramer <david@sentry.io>
Slim the guard to the tweet failure mode only: structured field dropped and
value survives under message/log.body. Keep validation repair renames working.

Co-Authored-By: David Cramer <david@sentry.io>
@sentry-junior sentry-junior Bot changed the title fix(search-events): reject silent full-text filter downgrades fix(search-events): reject message full-text filter downgrades Aug 15, 2026
Collapse the dual-prompt try/fail/retry loop. Give the agent a validateSearch
tool and one shared prompt. Handler keeps a single final validation gate and
still rejects structured-to-message downgrades.

Co-Authored-By: David Cramer <david@sentry.io>
@sentry-junior sentry-junior Bot changed the title fix(search-events): reject message full-text filter downgrades fix(search-events): stop silent full-text downgrades Aug 15, 2026
Comment thread packages/mcp-core/src/tools/support/search-events/utils.ts Outdated
Make semantic-downgrade detection quote-aware so literal message:/log.body:
text inside quoted values is not treated as a full-text filter rewrite.
Comment thread packages/mcp-core/src/tools/catalog/search-events.test.ts
Compare structured filters by key+value cardinality so dropping one of
several identical keys into message/log.body still counts as a semantic
downgrade.
@dcramer
dcramer marked this pull request as ready for review August 15, 2026 02:53

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 68df4d4. Configure here.

Comment thread packages/mcp-core/src/tools/support/search-events/utils.ts Outdated
Quoted multi-word values were truncated at the first space, so later
words could be rewritten into message/log.body without detection.
Comment thread packages/mcp-core/src/tools/support/search-events/utils.ts Outdated
Bare substring comparison treated unrelated full-text as a downgrade
(e.g. id:1 vs message:"error 401"). Require whole-token overlap instead.

@dcramer dcramer left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i want some evals that show search-events behavior as expected. it should be fairly light weight, use the msw stuff for the sentry api and really assert on outputs and inputs

Comment thread packages/mcp-core/src/tools/catalog/search-events.test.ts
Drop agent prompt assertions and document that policy. Cover the
structured-to-full-text false-success path with MSW-backed input/output
checks and light agent evals for expected query shape.
@sentry-junior

sentry-junior Bot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor Author

Addressed in 99bf5c6.

MSW I/O coverage (handler tests)

  • tweet path: agent rewrites conv_id:X → message:"*X*" → final validate keeps conv_id:X, events never called
  • partial duplicate-key downgrade kept + asserted on validate query
  • real rename still executes on the non-trusted path
  • agent-repair path asserts dataset/query/fields + formatted output (no prompt text)

Light agent evals

  • search-events-agent.eval.ts: structured conv_id must stay structured (no message/log.body rewrite)
  • real rename case for expected query shape

Policy

  • docs/testing/overview.md: do not assert agent/system prompt wording; assert tool I/O + Sentry API requests instead

@dcramer
dcramer merged commit 7a77cc5 into main Aug 16, 2026
21 checks passed
@dcramer
dcramer deleted the fix/search-events-reject-semantic-downgrade branch August 16, 2026 17:43

This branch was previously deployed

1 inactive deployment
Actions — 99bf5c66 Deployed Aug 15, 2026 by sentry-junior[bot] via eval #1087
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant