[BUG] [V2] Remote task interface rebuild fails with KeyError 'properties' when a $ref points to a non-object definition #8281

Summary

A task can't be launched by reference when one of its types has a list whose items $ref a $defs entry that is not an object. The client rebuilds the task's types from the registered JSON schema (TaskDetails.interface), and that rebuild raises KeyError: 'properties' before any run is created.

• Who hits it. Pydantic writes such a $defs entry for every named type alias (type X = ..., PEP 695), e.g. a URI string alias. Any task whose inputs or outputs contain list[X] of such an alias can't be launched with flyte.remote.Task.get(...) plus flyte.run.
• A second symptom. The same $ref on a plain field doesn't raise, but rebuilds as an empty DynamicModel instead of str.
• Versions. Seen on flyte 2.12.2 (the latest release). The same code is on main at f22c708a9e.

Reproduction

No cluster needed: the client runs the same rebuild that TaskDetails.interface runs on a fetched task.

from typing import Annotated

from flyte.types import TypeEngine
from pydantic import BaseModel, Field

type Uri = Annotated[str, Field(min_length=1)]  # named alias: pydantic emits $defs + $ref


class ListOfAlias(BaseModel):
    inputs: list[Uri]


class FieldOfAlias(BaseModel):
    source: Uri


TypeEngine.guess_python_type(TypeEngine.to_literal_type(ListOfAlias))
# KeyError: 'properties'

rebuilt = TypeEngine.guess_python_type(TypeEngine.to_literal_type(FieldOfAlias))
# no error, but rebuilt.source is an empty DynamicModel, not str

The schema pydantic gives ListOfAlias:

{"$defs": {"Uri": {"minLength": 1, "type": "string"}},
 "properties": {"inputs": {"items": {"$ref": "#/$defs/Uri"}, "title": "Inputs", "type": "array"}},
 "required": ["inputs"], "title": "ListOfAlias", "type": "object"}

The failing frames, in flyte/types/_type_engine.py:

782   _create_pydantic_model_from_schema       field_type = _get_pydantic_element_type(prop, schema)
719   _get_pydantic_element_type               return _get_element_type(element_property, schema)
2965  _get_element_type                        return typing.List[_get_element_type(element_property.get("items", {}), schema)]
2915  _get_element_type                        return convert_mashumaro_json_schema_to_python_class(ref_schema, ref_name)
2816  convert_mashumaro_json_schema_to_python_class
1562  generate_attribute_list_from_dataclass_json_mixin   properties = schema["properties"]   <- KeyError

A $ref to an object definition (a nested model) rebuilds fine, as does an enum.

Cause

Both $ref branches assume that a definition which is neither an enum nor a registered type is a model with properties.

• _get_element_type, L2898–L2917: it passes the definition to convert_mashumaro_json_schema_to_python_class, which reads schema["properties"] and raises. List items take this path.
• _get_pydantic_element_type, L673–L685: it passes the definition to _create_pydantic_model_from_schema, which builds a model with no fields. Plain fields take this path, so the wrong type goes unnoticed.

Pydantic writes a $defs entry for any named alias, whatever its target: a string, a number, a list or a union. JSON Schema allows that, so these schemas are valid.

Expected behaviour and a suggested fix

A $ref to a definition that is not an object should rebuild as that definition's own type. Here ListOfAlias.inputs should be list[str] and FieldOfAlias.source should be str.

The fix is the same in both branches: when the definition has no properties (and isn't an enum or a registered type), resolve the definition itself instead of building a model from it.

# _get_element_type, after the enum and registered-type checks
if "properties" not in ref_schema:
    return _get_element_type(ref_schema, schema)

# _get_pydantic_element_type, likewise
if "properties" not in ref_schema:
    return _get_pydantic_element_type(ref_schema, schema)

We checked this by patching both functions locally. The repro then rebuilds ListOfAlias.inputs as List[str] and FieldOfAlias.source as str, and models nested by $ref (inner: Inner, many: list[Inner]) rebuild as before. Happy to open a PR with tests.

Impact and workaround

We launch tasks by reference from a service that never imports the task code. Every output our tasks return carries a metadata model with a list[Ref] field, where Ref is a named alias. So none of these tasks could be launched: reading TaskDetails.interface raised the KeyError.

Two possible workarounds:

• In the model: annotate the field as list[Annotated[Ref[...], WithJsonSchema({...})]]. That writes the alias's schema in place, so pydantic emits no $defs entry for it.
• In the launching client: after fetching the task and before reading TaskDetails.interface, rewrite each input and output LiteralType.metadata. Any $ref to a $defs entry that has no properties (and isn't an enum) is replaced by that entry. This covers every alias without touching the models.

Environment

• flyte 2.12.2, the latest release; the same code is on main at f22c708a9e
• pydantic 2.13.4
• Python 3.12.13 on Linux