feat: checking span.name, span.description and gen_ai.operation.name

[constantinius]

Closes https://linear.app/getsentry/issue/TET-1662/expand-sdk-integration-tests-to-cover-everything-weve-talked-about
Closes https://linear.app/getsentry/issue/TET-1746/testing-framework-not-checking-value-of-gen-aioperationname

[linear]

TET-1662 Expand SDK integration tests to cover everything we've talked about

TET-1746 Testing framework not checking value of gen_ai.operation.name

[github-actions]

🔴 AI SDK Integration Test Results

Status: 31 regressions detected

Summary

Metric	main	PR	Change
Total Tests	159	159	—
Passed	49	18	-31 ⚠️
Failed	110	141	+31 ⚠️

🔴 Regressions

These tests were passing on main but are now failing:

py/openai-agents :: Basic Agent Test (async)

Error: 2 check(s) failed:

2 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'invoke_agent gpt-4o-mini' but is missing
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-4o-mini' but is missing

js/openai :: Basic LLM Test (blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing

js/openai :: Multi-Turn LLM Test (blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing
  Span 1: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing
  Span 2: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing

py/anthropic :: Basic LLM Test (sync, blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat claude-haiku-4-5' but is missing

py/anthropic :: Basic LLM Test (async, blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat claude-haiku-4-5' but is missing

py/anthropic :: Multi-Turn LLM Test (sync, blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat claude-haiku-4-5' but is missing
  Span 1: Attribute 'span.description' must equal 'chat claude-haiku-4-5' but is missing
  Span 2: Attribute 'span.description' must equal 'chat claude-haiku-4-5' but is missing

py/anthropic :: Multi-Turn LLM Test (async, blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat claude-haiku-4-5' but is missing
  Span 1: Attribute 'span.description' must equal 'chat claude-haiku-4-5' but is missing
  Span 2: Attribute 'span.description' must equal 'chat claude-haiku-4-5' but is missing

py/anthropic :: Vision LLM Test (sync, blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat claude-haiku-4-5' but is missing

py/anthropic :: Vision LLM Test (async, blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat claude-haiku-4-5' but is missing

py/langchain :: Basic LLM Test (sync, streaming)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing

py/langchain :: Basic LLM Test (sync, blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing

py/langchain :: Basic LLM Test (async, streaming)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing

py/langchain :: Basic LLM Test (async, blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing

py/langchain :: Multi-Turn LLM Test (sync, streaming)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing
  Span 1: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing
  Span 2: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing

py/langchain :: Multi-Turn LLM Test (sync, blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing
  Span 1: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing
  Span 2: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing

py/langchain :: Multi-Turn LLM Test (async, streaming)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing
  Span 1: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing
  Span 2: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing

py/langchain :: Multi-Turn LLM Test (async, blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing
  Span 1: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing
  Span 2: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing

py/langchain :: Vision LLM Test (sync, streaming)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-4o-mini' but is missing

py/langchain :: Vision LLM Test (sync, blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-4o-mini' but is missing

py/langchain :: Vision LLM Test (async, streaming)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-4o-mini' but is missing

py/langchain :: Vision LLM Test (async, blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-4o-mini' but is missing

py/litellm :: Basic LLM Test (sync, streaming)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing

py/litellm :: Basic LLM Test (sync, blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing

py/litellm :: Multi-Turn LLM Test (sync, streaming)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing
  Span 1: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing
  Span 2: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing

py/litellm :: Multi-Turn LLM Test (sync, blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing
  Span 1: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing
  Span 2: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing

py/litellm :: Vision LLM Test (sync, streaming)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-4o-mini' but is missing

py/litellm :: Vision LLM Test (sync, blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-4o-mini' but is missing

py/openai :: Basic LLM Test (sync, blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing

py/openai :: Basic LLM Test (async, blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing

py/openai :: Multi-Turn LLM Test (sync, blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing
  Span 1: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing
  Span 2: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing

py/openai :: Multi-Turn LLM Test (async, blocking)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 0: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing
  Span 1: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing
  Span 2: Attribute 'span.description' must equal 'chat gpt-5-nano' but is missing

Test Matrix

SDK	Basic Agent Test	Basic Agent Test (async)	Basic Agent Test (sync)	Basic Error LLM Test (async, blocking)	Basic Error LLM Test (async, streaming)	Basic Error LLM Test (blocking)	Basic Error LLM Test (streaming)	Basic Error LLM Test (sync, blocking)	Basic Error LLM Test (sync, streaming)	Basic LLM Test (async, blocking)	Basic LLM Test (async, streaming)	Basic LLM Test (blocking)	Basic LLM Test (streaming)	Basic LLM Test (sync, blocking)	Basic LLM Test (sync, streaming)	Long Input Agent Test	Long Input Agent Test (async)	Long Input Agent Test (sync)	Long Input LLM Test (async, blocking)	Long Input LLM Test (async, streaming)	Long Input LLM Test (blocking)	Long Input LLM Test (streaming)	Long Input LLM Test (sync, blocking)	Long Input LLM Test (sync, streaming)	Multi-Turn LLM Test (async, blocking)	Multi-Turn LLM Test (async, streaming)	Multi-Turn LLM Test (blocking)	Multi-Turn LLM Test (streaming)	Multi-Turn LLM Test (sync, blocking)	Multi-Turn LLM Test (sync, streaming)	Tool Call Agent Test	Tool Call Agent Test (async)	Tool Call Agent Test (sync)	Tool Error Agent Test	Tool Error Agent Test (async)	Tool Error Agent Test (sync)	Vision Agent Test	Vision Agent Test (async)	Vision Agent Test (sync)	Vision LLM Test (async, blocking)	Vision LLM Test (async, streaming)	Vision LLM Test (blocking)	Vision LLM Test (streaming)	Vision LLM Test (sync, blocking)	Vision LLM Test (sync, streaming)
js/anthropic	—	—	—	—	—	✅	✅	—	—	—	—	❌	❌	—	—	—	—	—	—	—	❌	❌	—	—	—	—	❌	❌	—	—	—	—	—	—	—	—	—	—	—	—	—	❌	❌	—	—
js/google-genai	—	—	—	—	—	❌	❌	—	—	—	—	❌	❌	—	—	—	—	—	—	—	❌	❌	—	—	—	—	❌	❌	—	—	—	—	—	—	—	—	—	—	—	—	—	❌	❌	—	—
js/langchain	—	—	—	—	—	✅	✅	—	—	—	—	❌	❌	—	—	—	—	—	—	—	❌	❌	—	—	—	—	❌	❌	—	—	—	—	—	—	—	—	—	—	—	—	—	❌	❌	—	—
js/mastra	✅	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	❌	—	—	❌	—	—	✅	—	—	—	—	—	—	—	—
js/openai	—	—	—	—	—	✅	✅	—	—	—	—	❌📉	❌	—	—	—	—	—	—	—	❌	❌	—	—	—	—	❌📉	❌	—	—	—	—	—	—	—	—	—	—	—	—	—	❌	❌	—	—
js/vercel	❌	—	—	—	—	—	—	—	—	—	—	—	—	—	—	❌	—	—	—	—	—	—	—	—	—	—	—	—	—	—	❌	—	—	❌	—	—	❌	—	—	—	—	—	—	—	—
py/anthropic	—	—	—	✅	❌	—	—	✅	❌	❌📉	❌	—	—	❌📉	❌	—	—	—	❌	❌	—	—	❌	❌	❌📉	❌	—	—	❌📉	❌	—	—	—	—	—	—	—	—	—	❌📉	❌	—	—	❌📉	❌
py/google-genai	—	❌	❌	—	—	—	—	—	—	—	—	—	—	—	—	—	❌	❌	—	—	—	—	—	—	—	—	—	—	—	—	—	❌	❌	—	❌	❌	—	❌	❌	—	—	—	—	—	—
py/langchain	—	—	—	✅	✅	—	—	✅	✅	❌📉	❌📉	—	—	❌📉	❌📉	—	—	—	❌	❌	—	—	❌	❌	❌📉	❌📉	—	—	❌📉	❌📉	—	—	—	—	—	—	—	—	—	❌📉	❌📉	—	—	❌📉	❌📉
py/langgraph	—	❌	❌	—	—	—	—	—	—	—	—	—	—	—	—	—	❌	❌	—	—	—	—	—	—	—	—	—	—	—	—	—	❌	❌	—	❌	❌	—	❌	❌	—	—	—	—	—	—
py/litellm	—	—	—	❌	❌	—	—	❌	❌	❌	❌	—	—	❌📉	❌📉	—	—	—	❌	❌	—	—	❌	❌	❌	❌	—	—	❌📉	❌📉	—	—	—	—	—	—	—	—	—	❌	❌	—	—	❌📉	❌📉
py/openai	—	—	—	✅	✅	—	—	✅	✅	❌📉	❌	—	—	❌📉	❌	—	—	—	❌	❌	—	—	❌	❌	❌📉	❌	—	—	❌📉	❌	—	—	—	—	—	—	—	—	—	❌	❌	—	—	❌	❌
py/openai-agents	—	❌📉	—	—	—	—	—	—	—	—	—	—	—	—	—	—	❌	—	—	—	—	—	—	—	—	—	—	—	—	—	—	❌	—	—	❌	—	—	❌	—	—	—	—	—	—	—
py/pydantic-ai	—	❌	—	—	—	—	—	—	—	—	—	—	—	—	—	—	❌	—	—	—	—	—	—	—	—	—	—	—	—	—	—	❌	—	—	❌	—	—	❌	—	—	—	—	—	—	—

Legend: ✅ Pass | ❌ Fail | ✅🔧 Fixed | ❌📉 Regressed | ✅🆕 New (pass) | ❌🆕 New (fail) | 🗑️ Removed

---

Generated by AI SDK Integration Tests