Skip to content

fix: preserve chat stream moderation results - #3864

Merged
marcuswood-oai merged 2 commits into
mainfrom
fix/preserve-chat-stream-moderation
Sep 16, 2026
Merged

marcuswood-oai merged 2 commits into
mainfrom
fix/preserve-chat-stream-moderation

Conversation

@marcuswood-oai

Copy link
Copy Markdown
Contributor

Preserves moderation results in chat stream snapshots and final completions, including reports delivered after content. Later omitted or null fields retain the last report; unmoderated streams remain unchanged.

Validation: 324 tests passed (14 skipped), plus 19 focused tests with Pydantic v1. Ruff, mypy, changed-file formatting, and wheel/sdist builds pass.

Carry moderation reports into accumulated completions while preserving the last report across omitted or null values. Add sync and async coverage for reports, errors, and unmoderated streams.
@marcuswood-oai
marcuswood-oai requested a review from a team as a code owner September 15, 2026 19:40
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 15, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-15T19:56:13.042807Z f9de450 New commits
🔒 Security Review ✅ Completed 2026-09-15T19:56:49.167708Z f9de450 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@openai-sdks

openai-sdks Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

OkTest Summary

❌ 234/236 SDK tests passed in 11.553s for Python SDK PR #3864.

Test results — 42 files
Test Result Time
tests/chat-completions-complex-body.test.ts ✅ Passed 179ms
tests/chat-completions-create.test.ts ✅ Passed 255ms
tests/chat-completions-stream.test.ts ✅ Passed 246ms
tests/files-content-binary.test.ts ✅ Passed 217ms
tests/files-create-multipart.test.ts ✅ Passed 217ms
tests/files-list-pagination.test.ts ✅ Passed 236ms
tests/initialize-config.test.ts ✅ Passed 200ms
tests/instance-isolation.test.ts ✅ Passed 199ms
tests/models-list.test.ts ✅ Passed 214ms
tests/responses-background-lifecycle.test.ts ✅ Passed 208ms
tests/responses-body-method-errors.test.ts ✅ Passed 320ms
tests/responses-cancel-timeout.test.ts ✅ Passed 195ms
tests/responses-cancel.test.ts ✅ Passed 204ms
tests/responses-compact-retries.test.ts ✅ Passed 287ms
tests/responses-compact.test.ts ✅ Passed 217ms
tests/responses-create-advanced-stream.test.ts ✅ Passed 107ms
tests/responses-create-advanced.test.ts ✅ Passed 235ms
tests/responses-create-disconnect.test.ts ✅ Passed 1.112s
tests/responses-create-errors.test.ts ✅ Passed 159ms
tests/responses-create-malformed-api-responses.test.ts ✅ Passed 432ms
tests/responses-create-retries.test.ts ✅ Passed 235ms
tests/responses-create-stream-failures.test.ts ❌ Failed 257ms
tests/responses-create-stream-timeout.test.ts ✅ Passed 234ms
tests/responses-create-stream-wire.test.ts ❌ Failed 3.816s
tests/responses-create-stream.test.ts ✅ Passed 430ms
tests/responses-create-terminal-states.test.ts ✅ Passed 249ms
tests/responses-create-timeout.test.ts ✅ Passed 201ms
tests/responses-create.test.ts ✅ Passed 218ms
tests/responses-delete.test.ts ✅ Passed 217ms
tests/responses-input-items-errors.test.ts ✅ Passed 292ms
tests/responses-input-items-list.test.ts ✅ Passed 294ms
tests/responses-input-items-options.test.ts ✅ Passed 357ms
tests/responses-input-tokens-count-timeout.test.ts ✅ Passed 280ms
tests/responses-input-tokens-count.test.ts ✅ Passed 297ms
tests/responses-malformed-inputs.test.ts ✅ Passed 2.831s
tests/responses-not-found-errors.test.ts ✅ Passed 235ms
tests/responses-parse.test.ts ✅ Passed 516ms
tests/responses-retrieve-retries.test.ts ✅ Passed 372ms
tests/responses-retrieve.test.ts ✅ Passed 323ms
tests/responses-stored-method-errors.test.ts ✅ Passed 579ms
tests/retry-behavior.test.ts ✅ Passed 3.195s
tests/sdk-error-shape.test.ts ✅ Passed 258ms

View OkTest run #35016395501

SDK merge (2ab8b8e1064b) · head (f9de450750b8) · base (9640f2903d65) · OkTest (08ec4299e769)

@github-actions

github-actions Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Castiron custom code

✅ No new custom-code files detected.

45 mixed files remain; 0 existing customizations changed.

Compared 9640f2903d65 → f9de450750b8. Generated baselines verified.

45 existing customizations unchanged
  • api.md
  • scripts/castiron/README.md
  • scripts/castiron/custom_code_report.py
  • scripts/castiron/test_custom_code_report.py
  • src/openai/init.py
  • src/openai/_client.py
  • src/openai/resources/audio/transcriptions.py
  • src/openai/resources/audio/translations.py
  • src/openai/resources/beta/agents/sessions/sessions.py
  • src/openai/resources/beta/beta.py
  • src/openai/resources/beta/responses/responses.py
  • src/openai/resources/beta/threads/runs/runs.py
  • src/openai/resources/beta/threads/threads.py
  • src/openai/resources/chat/completions/completions.py
  • src/openai/resources/embeddings.py
  • src/openai/resources/files.py
  • src/openai/resources/live/forks.py
  • src/openai/resources/live/live.py
  • src/openai/resources/live/sideband.py
  • src/openai/resources/realtime/realtime.py
  • src/openai/resources/responses/responses.py
  • src/openai/resources/uploads/uploads.py
  • src/openai/resources/vector_stores/file_batches.py
  • src/openai/resources/vector_stores/files.py
  • src/openai/resources/videos.py
  • src/openai/resources/webhooks/init.py
  • src/openai/resources/webhooks/webhooks.py
  • src/openai/types/beta/agent_session_message.py
  • src/openai/types/chat/init.py
  • src/openai/types/chat/chat_completion_message_tool_call.py
  • src/openai/types/fine_tuning/fine_tuning_job_integration.py
  • src/openai/types/realtime/conversation_item_input_audio_transcription_delta_event.py
  • src/openai/types/realtime/realtime_error_event.py
  • src/openai/types/responses/init.py
  • src/openai/types/responses/response.py
  • src/openai/types/responses/response_function_web_search.py
  • src/openai/types/responses/response_function_web_search_param.py
  • src/openai/types/responses/responses_client_event.py
  • src/openai/types/responses/responses_client_event_param.py
  • src/openai/types/responses/tool.py

5 more in the full report.

A changed generated baseline means this report cannot reliably identify which handwritten lines changed.

Inspect the custom-code diff

Download the exact patch produced by this run (requires repository access):

gh run download 35016545074 --repo openai/openai-python \
  --name castiron-custom-code-35016545074-1 --dir /tmp/castiron-custom-code-35016545074-1
git apply --stat /tmp/castiron-custom-code-35016545074-1/custom-code.patch
cat /tmp/castiron-custom-code-35016545074-1/custom-code.patch

Or reproduce it from an SDK checkout containing the vendored reporter:

git fetch --no-tags origin 9640f2903d65c43f0e5aee58d60f46880ff0bd15 f9de450750b86f0b21fb032508e27410e77af789
python3 scripts/castiron/custom_code_report.py report \
  --base 9640f2903d65c43f0e5aee58d60f46880ff0bd15 \
  --head f9de450750b86f0b21fb032508e27410e77af789 --fetch --require-head-hash --public \
  --out /tmp/castiron-custom-code-f9de450750b8
cat /tmp/castiron-custom-code-f9de450750b8/custom-code.patch

This is the current full custom patch for mixed files, not an attribution of only the handwritten lines changed by this PR.

Full report and patch

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the accumulator and the sync/async regression matrix. Reconstructing the chunk-side moderation value as the completion-side generated model is the right type boundary; later non-null reports replace the snapshot, omitted/null moderation preserves the last report, and the initial-chunk conversion already carries an initial report. Coverage exercises success, both input/output errors, replacement, late delivery after finish, unmoderated streams, raw chunks, snapshots, final completions, and Pydantic v1. No blocking issues found.

@markstuart-oai markstuart-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 65ba4ab1825d29377f141d226edb36990c3f7bb1. No findings. The accumulator converts chunk-side moderation into the completion-side model, retains it through omitted/null chunks, and replaces it only when a new report arrives. Initial, late, successful and error reports remain consistent across sync/async snapshots and final completions. The conversion also isolates the snapshot from later mutation of the raw chunk while preserving extra fields. The change stays in the existing accumulation path without adding another abstraction.

Validation: 49 existing streaming/moderation tests passed, plus eight independent public-stream cases, two blocking-moderation controls and a snapshot-isolation probe. Exact-base controls demonstrate the late/replacement fix. Current CI passed Python 3.10/3.14, HTTPX2, lint and build checks. Local checks used Pydantic v2; a local v1 environment was unavailable. No live API calls were made.

@marcuswood-oai
marcuswood-oai added this pull request to the merge queue Sep 16, 2026
Merged via the queue into main with commit b77076d Sep 16, 2026
26 checks passed
@marcuswood-oai
marcuswood-oai deleted the fix/preserve-chat-stream-moderation branch September 16, 2026 16:05
@openai-sdks openai-sdks Bot mentioned this pull request Sep 16, 2026
pull Bot pushed a commit to Mattlk13/openai-python that referenced this pull request Sep 18, 2026
Automated Release PR
---


##
[3.15.0](openai/openai-python@v3.14.1...v3.15.0)
(2026-09-18)


### Features

* **api:** add agent session model settings
([openai#3882](openai#3882))
([4b15817](openai@4b15817))
* **api:** add audio-mini model choices
([openai#3886](openai#3886))
([a6eeb3f](openai@a6eeb3f))
* **api:** add compaction progress events
([openai#3866](openai#3866))
([98e1d24](openai@98e1d24))
* **api:** add managed Responses WebSocket sessions
([openai#3887](openai#3887))
([3b865af](openai@3b865af))
* **api:** add prompt-cache prewarming
([openai#3888](openai#3888))
([8f7cdb8](openai@8f7cdb8))


### Bug Fixes

* preserve chat stream moderation results
([openai#3864](openai#3864))
([b77076d](openai@b77076d))


### Chores

* **api:** clarify incoming SIP call ID usage
([openai#3885](openai#3885))
([f0457ab](openai@f0457ab))
* **api:** update image request examples
([openai#3889](openai#3889))
([bb79676](openai@bb79676))


### Documentation

* **realtime:** remove nonexistent type imports
([openai#3869](openai#3869))
([2971aaf](openai@2971aaf))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: openai-sdks[bot] <284451331+openai-sdks[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants