Introspection: Introduce TypeHint struct #5438

Tpt · 2025-09-12T13:00:37Z

inspect::TypeHint is an AST of a Python type hint.

It is supposed to be created using a set of const constructors and used only in const context.

Introspection data generation is done using two associated methods. The serialized data is also an AST.

I make use of the AST to generate nice from X import Y as Z lines in the type stubs with code that auto-detect conflicts.

Miscellaneous changes:

Rename PyType{Info,Check}::TYPE_INFO into TYPE_HINT
Drop redundant PyClassImpl::TYPE_NAME

Tpt · 2025-09-12T13:25:57Z

src/inspect/mod.rs

+pub struct TypeHint {
+    /// The type hint annotation
+    #[doc(hidden)]
+    pub annotation: &'static str,


I would have preferred it to be an enum but sadly I don't see how to serialize it: const functions cannot concatenate and recursive macros are not possible

What would the enum definition be? I can see if I can come up with any crazy ideas if I know the preferred form.

Something like:

enum TypeHint {clashes BuiltIn { name: &'static str }, Member { module: &'static str, name: &'static str }, Union(&[TypeHint]), Subscript { value: TypeHint, parameters: &[TypeHint] }, Callable { arguments: &[TypeHint], return: TypeHint } // Special case because of the nested [] syntax }

This way it's easy to ensure the syntax is valid and to do manipulation like introducing aliases in case of name conflicts.

Hmm, I think if we split the const fn up into one which (recursively) calculates the final length, and a second which serializes to it with &mut [u8] similar to what I did in the concat.rs module, I think we could probably make the const fn work?

Unfortunately this would require MSRV 1.83. But this is currently an experimental feature, I would personally be ok with having it limited to MSRV 1.83.

Done!

This works fairly well. I have not introduced macros to build the TypeHint enum yet, I plan to do it in a follow up. My guess is that the real use case is for the custom type hint in the signature, most of INPUT_TYPE and OUTPUT_TYPE are likely fine with just the const constructors.

I now serialize the type hint as an AST, this allows to manipulate it when building the type stubs. As an example, I wrote some code that generates nice from X import Y imports and generate aliases in case of name conflicts.

Tpt · 2025-09-12T13:26:23Z

src/inspect/mod.rs

+/// assert_eq!(T.to_string(), "collections.abc.Sequence");
+/// ```
+#[macro_export]
+macro_rules! type_hint {


Might be better as a const function. WDYT?

I wonder, is there a way to merge all the macros into one? Maybe also with implicit syntax

type_hint!(a.A); // basic type hint type_hint!(a.A | b.B); // union type hint type_hint!(a.A[b.B]); // parameterised type hint

we could infer all the imports using the dotted path rules, maybe in complex cases we require users to write the imports?

type_hint!( from a import A; from b import B; A.NestedClass | B );

or maybe can use mixed syntax somehow

type_hint!(type_hint("a", "A.NestedClass") | b.B);

the "implicit syntax" seems nicest for the 99% of cases which users will write, so while I'm not sure exactly how to tie it all together it seems desirable to me (I think?) to have an intuitive syntax for the common case and then an escape hatch for the awkward ones.

This looks indeed much nicer. I think we need a syntax for composition like in #[derive(FromPyObject)]: we want their to build type hints that is stuff like the union of type hints. So we want to write things like type_hint!(tuple[A::INPUT_TYPE, B::INPUT_TYPE]). But my guess it's mostly for auto-generated code.

What about having two macros:

For static type hints like type_hint!(a.A[b.B]); with hopefully support for the from a import A syntax.

For composition compose_type_hint!() where elements are all supposed to be other type hints like compose_type_hint!(PyTuple::INPUT_TYPE [ A::INPUT_TYPE, B::INPUT_TYPE ]). But I am not sure it's more readable than the current state.

WDYT?

It might also take into account in this design the type stubs in signatures like signature = (foo: collections.abs.Sequence[int]) -> list[int]. My guess is that we can allow there the same syntax than type_hint! and allow to set the from a import A in #[pymodule]. But it makes name conflict detection more fun...

davidhewitt

Thanks, this looks like the right direction for making the type hint definitions reliable wrt import resolution.

I think we can iterate a bit to find the best design for users, I wrote down some ideas.

davidhewitt · 2025-09-14T11:44:29Z

src/inspect/mod.rs

+    }};
+}
+
+/// Allows to build a [`TypeHint`] that is the subscripted


Sentence looks unfinished.

davidhewitt · 2025-09-14T11:55:06Z

src/inspect/mod.rs

+pub struct TypeHint {
+    /// The type hint annotation
+    #[doc(hidden)]
+    pub annotation: &'static str,


What would the enum definition be? I can see if I can come up with any crazy ideas if I know the preferred form.

davidhewitt · 2025-09-14T12:00:22Z

src/inspect/mod.rs

+/// assert_eq!(T.to_string(), "collections.abc.Sequence");
+/// ```
+#[macro_export]
+macro_rules! type_hint {


I wonder, is there a way to merge all the macros into one? Maybe also with implicit syntax

type_hint!(a.A); // basic type hint type_hint!(a.A | b.B); // union type hint type_hint!(a.A[b.B]); // parameterised type hint

we could infer all the imports using the dotted path rules, maybe in complex cases we require users to write the imports?

type_hint!( from a import A; from b import B; A.NestedClass | B );

or maybe can use mixed syntax somehow

type_hint!(type_hint("a", "A.NestedClass") | b.B);

the "implicit syntax" seems nicest for the 99% of cases which users will write, so while I'm not sure exactly how to tie it all together it seems desirable to me (I think?) to have an intuitive syntax for the common case and then an escape hatch for the awkward ones.

davidhewitt · 2025-09-14T12:04:49Z

pyo3-introspection/src/introspection.rs

It looks like the annotation_stub stuff is still present, should we remove it at the same time?

The code to extract modules was in the stubs.rs module and got removed

inspect::TypeHint is composed of an "annotation" string and a list of "imports" ("from X import Y" kind) The type is expected to be built using the macros `type_hint!(module, name)`, `type_hint_union!(*args)` and `type_hint_subscript(main, *args)` that take care of maintaining the import list Introspection data generation is done using the hidden type_hint_json macro to avoid that the proc macros generate too much code Sadly, outside `type_hint` these macros can't be converted into const functions because they need to do some concatenation. I introduced `type_hint!` for consistency, happy to convert it to a const function. Miscellaneous changes: - Rename PyType{Info,Check}::TYPE_INFO into TYPE_HINT - Drop redundant PyClassImpl::TYPE_NAME

Tpt · 2025-09-23T14:30:08Z

pyo3-introspection/src/model.rs

+#[derive(Debug, Eq, PartialEq, Clone, Hash)]
+pub enum TypeHint {
+    Ast(TypeHintExpr),
+    Plain(String),


This is for custom type hint witten in the #[pyo3(signature = ] attribute.

Tpt · 2025-09-23T14:39:37Z

Cargo.toml

    "chrono-tz",
    "either",
    "experimental-async",
-    "experimental-inspect",


I now added it dynamically in the _get_feature_sets function of the noxfile. Not sure if it's the best way to do it

Tpt · 2025-09-24T09:41:21Z

src/type_object.rs

+    /// Provides the full python type as a type hint.
    #[cfg(feature = "experimental-inspect")]
-    const PYTHON_TYPE: &'static str = "typing.Any";
+    const TYPE_HINT: TypeHint = TypeHint::module_member("typing", "Any");


I think we can drop this and only have the field on PyTypeCheck. I will do it in a follow up except if you prefer it done in this MR

Tpt changed the title ~~Introduce TypeHint struct~~ Introspection: Introduce TypeHint struct Sep 12, 2025

Tpt force-pushed the tpt/moduleandtype branch 3 times, most recently from 7d6b267 to 7bb5b56 Compare September 12, 2025 13:24

Tpt commented Sep 12, 2025

View reviewed changes

Tpt force-pushed the tpt/moduleandtype branch 5 times, most recently from 47a4fc1 to 4ece557 Compare September 12, 2025 14:15

Tpt marked this pull request as ready for review September 12, 2025 15:01

davidhewitt reviewed Sep 14, 2025

View reviewed changes

davidhewitt mentioned this pull request Sep 14, 2025

fill PyTypeInfo::NAME with Pythonic values (e.g. BaseException) instead of C ones (e.g. PyBaseException) #5352

Open

Tpt force-pushed the tpt/moduleandtype branch from 4ece557 to 02ce980 Compare September 23, 2025 14:23

Tpt commented Sep 23, 2025

View reviewed changes

Tpt force-pushed the tpt/moduleandtype branch from 02ce980 to 1f56ea9 Compare September 23, 2025 14:31

Makes TypeHint an enum

3fee134

Tpt force-pushed the tpt/moduleandtype branch from 1f56ea9 to 3fee134 Compare September 23, 2025 14:38

Tpt commented Sep 23, 2025

View reviewed changes

Tpt commented Sep 24, 2025

View reviewed changes

Introspection: Introduce TypeHint struct #5438

Are you sure you want to change the base?

Introspection: Introduce TypeHint struct #5438

Uh oh!

Conversation

Tpt commented Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Tpt Sep 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Tpt Sep 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Tpt Sep 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

davidhewitt left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Tpt commented Sep 12, 2025 •

edited

Loading

Tpt Sep 15, 2025 •

edited

Loading

Tpt Sep 23, 2025 •

edited

Loading

Tpt Sep 15, 2025 •

edited

Loading