Skip to content

Classify gpt-6-astra as reasoning + text/image-capable in OpenAI model metadata - #54

Draft
jeffpaul with Copilot wants to merge 3 commits into
trunkfrom
copilot/update-gpt-6-astra-classification
Draft

jeffpaul with Copilot wants to merge 3 commits into
trunkfrom
copilot/update-gpt-6-astra-classification

Conversation

Copilot AI commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

After 1.2.0, gpt-6-astra was discovered but classified as text-only, so clients requiring text + image input filtered it out. The root cause was model-ID patterning that did not include GPT-6 Astra in multimodal/reasoning capability paths.

  • Reasoning model detection

    • Extended reasoning ID matching to include gpt-6-astra and dated snapshots (e.g. gpt-6-astra-YYYY-MM-DD).
    • Ensures Astra follows reasoning-model option behavior (including no unsupported sampling options by default).
  • Input modality mapping (narrow Astra-specific path)

    • Added an Astra-specific classification branch for text + image input only.
    • Avoids reusing the generic multimodal option matrix that also advertises audio/document combinations not documented for Astra.
  • Focused metadata coverage

    • Added test cases asserting:
      • gpt-6-astra is treated as a reasoning model for sampling-option classification.
      • Astra input modalities resolve to exactly text and text+image.
private const REASONING_MODEL_ID_PATTERN =
    '/^(?:codex-mini-latest|gpt-5(?:\.\d+)?|gpt-6-astra(?:-\d{4}-\d{2}-\d{2})?|o(?:1|3|4))(?:-|$)/';

private static function supportsTextAndImageInputOnly(string $modelId): bool
{
    return (bool) preg_match('/^gpt-6-astra(?:-\d{4}-\d{2}-\d{2})?$/', $modelId);
}

Co-authored-by: jeffpaul <2818133+jeffpaul@users.noreply.github.com>
Copilot AI changed the title [WIP] Update OpenAI provider's model capability metadata for GPT-6 Astra Classify gpt-6-astra as reasoning + text/image-capable in OpenAI model metadata Sep 24, 2026
Copilot AI requested a review from jeffpaul September 24, 2026 01:53
@jeffpaul

Copy link
Copy Markdown
Member

@jeffpaul jeffpaul added this to the 1.3.0 milestone Sep 24, 2026

@saarnilauri saarnilauri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked a live /v1/models listing. Of 140 models, three are GPT-6:

gpt-6-astra
gpt-6-luna
gpt-6-sol

Through this PR's parser, gpt-6-astra resolves to text | text+image with no sampling options, as intended. gpt-6-luna and gpt-6-sol both resolve to text-only input with sampling options enabled, which is where Astra was before this PR.

OpenAI's latest-model guide says:

"All latest OpenAI models support text and image input, text output, multilingual capabilities, and vision."

"GPT-6 Astra does not support the none reasoning effort" … "GPT-6 Sol and Luna do."

So Astra is handled correctly: it's a reasoning model, it gets no sampling options because it can't use reasoning_effort: none, and it takes text + image input rather than the full audio/document set. Luna and Sol are also reasoning models with text + image input, but this PR leaves them classified as non-reasoning and text-only. The PR description gives the root cause as "model-ID patterning that did not include GPT-6 Astra in multimodal/reasoning capability paths". I think the patterning is missing GPT-6 as a whole, and Astra is the model that happened to get reported.

I'd suggest matching the generation instead of one model ID:

private const GPT6_MODEL_ID_PATTERN = '/^gpt-6-(?:astra|luna|sol)(?:-|$)/';

and using it in both REASONING_MODEL_ID_PATTERN and the new modality helper. With that change applied locally, all three models resolve to text | text+image and Astra still gets no sampling options.

One thing I couldn't settle from the docs: they say Sol and Luna support reasoning_effort: none, not that they default to it. EFFORT_NONE_DEFAULT_MODEL_ID_PATTERN is about the default, and that's what decides whether Sol and Luna should advertise sampling options. We should figure out their default?

Details inline.

Comment thread src/Metadata/OpenAiModelMetadataDirectory.php Outdated
&& !str_contains($modelId, '-transcribe')
) {
if (self::supportsMultimodalTextInput($modelId)) {
if (self::supportsTextAndImageInputOnly($modelId)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only matters once line 424 is widened, so please treat it as part of that change.

This branch runs before the supportsMultimodalTextInput() branch, which means it also runs before the -audio / -search handling nested at lines 361–364. Right now that's fine, because the $ anchor on line 424 keeps any suffixed ID from reaching this point.

If the pattern is widened to the prefix form suggested on line 424, a GPT-6 ID with a -search or -audio suffix would match here first and get $gptReasoningTextAndImageInputOptions, skipping $gptSearchOptions / $gptMultimodalSpeechOutputOptions. I'm not saying such an ID exists today. Lines 361–364 exist because other families ship those suffixes, and the handling should apply regardless of family.

Since the suffix checks apply across families, they could move ahead of the family dispatch instead of being nested inside one branch:

if (str_contains($modelId, '-audio')) {
    // ... existing audio handling
} elseif (str_contains($modelId, '-search')) {
    // ... existing search handling
} elseif (self::supportsTextAndImageInputOnly($modelId)) {
    // ... GPT-6
} elseif (self::supportsMultimodalTextInput($modelId)) {
    // ... existing multimodal handling
}

A regression test with a synthetic -search GPT-6 ID would keep this from coming back.

*/
private static function supportsTextAndImageInputOnly(string $modelId): bool
{
return (bool) preg_match('/^gpt-6-astra(?:-\d{4}-\d{2}-\d{2})?$/', $modelId);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking. This matches a single model ID, and the rest of the generation is left out. /v1/models currently lists three GPT-6 models, and this PR's parser gives:

model id input modalities sampling options
gpt-6-astra text | text+image no
gpt-6-luna text yes
gpt-6-sol text yes

Luna and Sol end up where Astra was before this PR. With text-only input, a client that requires text + image filters them out. And since neither matches REASONING_MODEL_ID_PATTERN, both advertise temperature/top_p/logprobs/top_logprobs.

According to OpenAI's latest-model guide, both of those are wrong:

"All latest OpenAI models support text and image input, text output, multilingual capabilities, and vision."

"GPT-6 Astra does not support the none reasoning effort" … "GPT-6 Sol and Luna do."

All three are reasoning models with text + image input. The PR handles Astra correctly, including leaving out sampling options since Astra can't use reasoning_effort: none. Luna and Sol need the same classification.

The PR description calls the root cause "model-ID patterning that did not include GPT-6 Astra in multimodal/reasoning capability paths." I'd describe it as the patterning not including GPT-6 at all. Astra is just the one that was reported.

Suggestion: a generation-level constant shared by isReasoningModel() and the modality helper:

/**
 * Regular expression matching the model IDs of the GPT-6 generation.
 *
 * @since n.e.x.t
 *
 * @var string
 */
private const GPT6_MODEL_ID_PATTERN = '/^gpt-6-(?:astra|luna|sol)(?:-|$)/';

With this applied locally, all three resolve to text | text+image, and Astra still has no sampling options.

Open question. The docs say Sol and Luna support reasoning_effort: none. They don't say either one defaults to it. EFFORT_NONE_DEFAULT_MODEL_ID_PATTERN is specifically about the default ("use reasoning effort none by default"), so I can't tell whether Sol and Luna belong there, and that's what decides whether they advertise sampling options. It would be good to confirm this before merge. It's the only claim in this comment I don't have a source for.

Also on this line. This pattern is exact-anchored ($), while REASONING_MODEL_ID_PATTERN on line 48, also edited in this PR, is prefix-anchored ((?:-|$)). Line 48 treats any gpt-6-astra-* as a reasoning model; line 424 only treats the bare and dated IDs as Astra. No suffixed Astra IDs exist today, so nothing breaks yet, but the two patterns should agree on what the family is.

The existing test data suggests the prefix form: gpt-5-mini, gpt-5.4-mini, gpt-5-pro, gpt-5.2-pro, gpt-5.4-nano, gpt-5.2-codex, gpt-5-chat-latest, and gpt-5.4-mini-2026-03-17. That last one is a variant plus a date, and an exact anchor like this one wouldn't match it.

Related: gpt-6-astra-2026-09-01 isn't in the live listing, so the dated-snapshot handling here (and its test case on line 99) is speculative for now. It does no harm, but the date group on line 48 is redundant either way (see my comment there).

If you relax this anchor, please also look at my comment on line 352. That change needs to go in at the same time.

),
new SupportedOption(OptionEnum::outputModalities(), [[ModalityEnum::text()]]),
]);
$gptTextAndImageInputOptions = array_merge($gptBaseOptions, $gptSamplingOptions, [

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please read this together with my comment on line 424 before deleting anything.

As the PR stands, this array is never selected. For every ID where supportsTextAndImageInputOnly() returns true:

  • isReasoningModel() is true (line 48 now includes gpt-6-astra)
  • isNonReasoningChatModel() is false (/^gpt-5-chat-latest$/)
  • hasDefaultReasoningEffortNone() is false (/^gpt-(?:5\.[12]|5\.4(?:-(?:mini|nano))?)…$/)

so supportsSamplingOptions() is always false, and the ternary on lines 354–356 always picks $gptReasoningTextAndImageInputOptions. The data provider in this PR asserts the same thing on lines 98–99 ('gpt-6-astra' => false).

It's unreachable because the PR only handles Astra, which is the one GPT-6 model that can't use reasoning_effort: none. According to the docs, Sol and Luna can. Once the generation is covered as suggested on line 424, a text + image model that also advertises sampling options is exactly what this array is for. I checked locally, and Luna and Sol end up here.

So there are two consistent options:

  • Cover the generation on line 424: keep this array. The ternary on lines 354–356 then actually does something.
  • Keep the PR scoped to Astra only: delete this array (162–171), its use capture (line 296), and the ternary, and assign $modelOptions = $gptReasoningTextAndImageInputOptions; directly. Unreachable code suggests behavior the code doesn't have.

The current state, where the array looks intentional but nothing can reach it, is the one to avoid.

Separately: this method now has three pairs of option arrays with the same structure ($gptOptions/$gptReasoningOptions, $gptMultimodalInputOptions/$gptReasoningMultimodalInputOptions, and the new pair). They differ only in whether $gptSamplingOptions is merged in and which modality list is used, and the use clause is up to 17 variables. This PR doesn't need to fix that, but if the modality list came from a per-family helper, each pair would collapse into one expression:

array_merge(
    $gptBaseOptions,
    self::supportsSamplingOptions($modelId) ? $gptSamplingOptions : [],
    [
        new SupportedOption(OptionEnum::inputModalities(), self::inputModalitiesFor($modelId)),
        new SupportedOption(OptionEnum::outputModalities(), [[ModalityEnum::text()]]),
    ]
);

*/
private const REASONING_MODEL_ID_PATTERN = '/^(?:codex-mini-latest|gpt-5(?:\.\d+)?|o(?:1|3|4))(?:-|$)/';
private const REASONING_MODEL_ID_PATTERN =
'/^(?:codex-mini-latest|gpt-5(?:\.\d+)?|gpt-6-astra(?:-\d{4}-\d{2}-\d{2})?|o(?:1|3|4))(?:-|$)/';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The (?:-\d{4}-\d{2}-\d{2})? group added after gpt-6-astra has no effect here. The alternation is followed by (?:-|$), so gpt-6-astra on its own already matches anything that starts with gpt-6-astra-, including gpt-6-astra-2026-09-01. I checked, and classification is the same with or without the group.

It also makes the pattern look like only bare and dated Astra IDs count as reasoning models. That's the assumption behind the mismatch I mentioned on line 424, so the group is misleading as well as redundant.

For comparison, the same group is needed in EFFORT_NONE_DEFAULT_MODEL_ID_PATTERN (line 61) because that pattern ends in $. And gpt-5(?:\.\d+)? is needed here because . isn't -, so (?:-|$) wouldn't match gpt-5.2.

I'd drop the group, which also lets the constant fit on one line again:

private const REASONING_MODEL_ID_PATTERN = '/^(?:codex-mini-latest|gpt-5(?:\.\d+)?|gpt-6-astra|o(?:1|3|4))(?:-|$)/';

(If you go with the generation-level constant from line 424, this line would reference that instead.)

Co-authored-by: Lauri Saarni <36833033+saarnilauri@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants