diff --git a/param/parameterized.pyi b/param/parameterized.pyi new file mode 100644 index 000000000..8142d2ccd --- /dev/null +++ b/param/parameterized.pyi @@ -0,0 +1,9 @@ +from typing import Any, Optional, overload + +class String: + default: Optional[str] + def __get__(self, instance: Any, owner: Any) -> Optional[str]: ... + def __set__(self, instance: Any, value: Optional[str]) -> None: ... + def __init__(self, default: Optional[str] = ..., doc: Optional[str] = ...) -> None: ... + +class ParameterizedMetaclass(type): ... \ No newline at end of file diff --git a/param/parameters.pyi b/param/parameters.pyi new file mode 100644 index 000000000..f98fde187 --- /dev/null +++ b/param/parameters.pyi @@ -0,0 +1,13 @@ +from typing import Any, Optional + +class Number: + default: float + def __get__(self, instance: Any, owner: Any) -> float: ... + def __set__(self, instance: Any, value: float) -> None: ... + def __init__(self, default: Optional[float] = ..., doc: Optional[str] = ...) -> None: ... + +class String: + default: str + def __get__(self, instance: Any, owner: Any) -> str: ... + def __set__(self, instance: Any, value: str) -> None: ... + def __init__(self, default: Optional[str] = ..., doc: Optional[str] = ...) -> None: ... \ No newline at end of file diff --git a/static-typing/analysis.md b/static-typing/analysis.md new file mode 100644 index 000000000..2c8bad4a4 --- /dev/null +++ b/static-typing/analysis.md @@ -0,0 +1,44 @@ +# Param - Static Typing - Analysis + +This document fixes the problem described in [problem.md](problem.md). + +## Updated Findings (2025) + +### Param Descriptors and Typing +- Param descriptors (e.g., param.String, param.Number) can provide good attribute type inference in mypy if their stubs define correct __get__ return types (e.g., str, Optional[str], float). +- Type annotations on descriptor assignments cause mypy errors unless suppressed with # type: ignore[assignment]. +- Wrapper functions using @overload (e.g., param_string) can help mypy infer the type of the value returned by the function, but the attribute is still a descriptor, so __init__ signatures are not synthesized. +- Mypy does not automatically add Param attributes to the __init__ signature, even with @dataclass_transform or wrapper functions. Only dataclasses, attrs, and pydantic are special-cased for this behavior. +- For best results, use stubs for descriptors and, if needed, class-specific stubs for correct __init__ signatures. + +### Wrapper Functions with @overload +- Providing wrapper functions like param_string with @overload can improve type inference for value-based APIs. +- For descriptor-based APIs like Param, wrapper functions do not solve the main static typing challenge: mypy cannot synthesize the correct __init__ signature or fully understand dynamic descriptor behavior. +- Therefore, wrapper functions are not recommended as a general solution for Param static typing. They may be useful for value-oriented usage, but not for descriptor-based classes. + +## Potential Solutions + +### 1. PEP 681 – Data Class Transforms +PEP 681’s @dataclass_transform helps type checkers recognize dataclass-like behavior, but does not synthesize __init__ signatures for Param-based classes automatically. Stubs or code generation are still needed for full support. + +### 2. Custom Type Checker Plugins +Custom plugins can provide better support but require ongoing maintenance and are less future-proof. + +### 3. Type Annotations via Code Generation or Metaclass Magic +Dynamic type hint generation is possible but complex and hard to maintain. + +### 4. Integration with Dataclasses or Pydantic +Conversion utilities can help users who need full static typing. + +### 5. Documentation and IDE Integration +Improved documentation and stubs help users get better type support. + +--- + +## Recommendations +- Adopt PEP 681’s @dataclass_transform for Param’s metaclass or base class. +- Provide and maintain high-quality stubs for Param descriptors. +- Use class-specific stubs for full static typing, including __init__ signatures. +- Consider conversion utilities to dataclasses or Pydantic for users needing strict typing. +- Avoid custom plugins unless absolutely necessary. +- Do not recommend wrapper functions with @overload as a general solution for Param static typing. \ No newline at end of file diff --git a/static-typing/experiment.md b/static-typing/experiment.md new file mode 100644 index 000000000..d602cabc7 --- /dev/null +++ b/static-typing/experiment.md @@ -0,0 +1,17 @@ +# Param - Static Typing - Experiment Results + +## experiment.py Summary + +The experiment demonstrates how mypy infers types for Param-based classes and wrapper function-based APIs: + +- Param descriptors (e.g., param.String, param.Number) with good stubs allow mypy to infer correct types for instance attributes (e.g., str, Optional[str], float). +- Wrapper functions using @overload (e.g., param_string) can help mypy infer the type of the value returned by the function, but the attribute is still a descriptor, so __init__ signatures are not synthesized. +- Type annotations on descriptor assignments cause mypy errors unless suppressed with # type: ignore[assignment]. +- Mypy does not automatically add Param attributes to the __init__ signature, even with @dataclass_transform or wrapper functions. Only dataclasses, attrs, and pydantic are special-cased for this behavior. +- For best results, use stubs for descriptors and, if needed, class-specific stubs for correct __init__ signatures. + +## Key Findings +- Param descriptors can provide good attribute type inference with stubs. +- Wrapper functions with @overload help with value type inference, but not with __init__ signatures. +- Mypy does not synthesize __init__ signatures for Param-based classes automatically. +- Class-specific stubs are required for full static typing support, including constructor signatures. diff --git a/static-typing/experiment.py b/static-typing/experiment.py new file mode 100644 index 000000000..c44e90954 --- /dev/null +++ b/static-typing/experiment.py @@ -0,0 +1,66 @@ +from typing import Any +from typing_extensions import dataclass_transform, reveal_type +import param +from typing import overload, Optional + +@dataclass_transform(field_specifiers=(param.String, param.Number)) +class PatchedParameterizedMetaclass(param.parameterized.ParameterizedMetaclass): + pass + +@overload +def param_string(default: str, doc: Optional[str] = None) -> str: ... +@overload +def param_string(default: None = None, doc: Optional[str] = None) -> Optional[str]: ... + +def param_string(default=None, doc=None): + return param.String(default=default, doc=doc) + +def test_string(default: str, doc=None)->str: + return default + +class MyClass(param.Parameterized, metaclass=PatchedParameterizedMetaclass): + my_parameter = param.String(default="some value", doc="A custom parameter") + my_optional_parameter = param.String(doc="A custom parameter with None as default") + my_default_is_None_parameter = param.String(default=None, doc="A custom parameter with None as default") + # my_parameter_annotated: str = param.String(default="annotated value", doc="A custom annotated parameter") + + my_param_string = param_string(default="default value", doc="A custom string parameter") + my_pyram_string_none = param_string(default=None, doc="A custom string parameter with None as default") + my_param_string_no_default = param_string(doc="A custom string parameter with no default") + my_param_string_annotated: str = param_string(default="annotated value", doc="A custom annotated string parameter") + + my_test_string = test_string("test", doc="A test string parameter") + my_number = param.Number(default=42.0, doc="A number parameter") + my_value_annotated: int = 0 # Example of a simple parameter + +def test_myclass() -> None: + reveal_type(MyClass.my_parameter) # Should be "builtins.str" + reveal_type(MyClass.my_optional_parameter) # Should be "typing.Optional[str]" + reveal_type(MyClass.my_default_is_None_parameter) # Should be "typing.Optional[str]" + + reveal_type(MyClass.my_param_string) # Should be "builtins.str" + reveal_type(MyClass.my_pyram_string_none) # Should be "typing.Optional[str]" + reveal_type(MyClass.my_param_string_no_default) # Should be "typing.Optional[str + + reveal_type(MyClass.my_test_string) # Should be "builtins.str" + reveal_type(MyClass.my_number) # Should be "builtins.float" + reveal_type(MyClass.my_value_annotated) # Should be "builtins.int" + + reveal_type(MyClass.__init__) # Should show correct signature + + obj = MyClass(my_parameter="hello", my_number=3.14) + + reveal_type(obj.my_parameter) # Should be "builtins.str" + reveal_type(obj.my_optional_parameter) # Should be "typing.Optional[str]" + reveal_type(obj.my_default_is_None_parameter) # Should be "typing.Optional[str]" + + reveal_type(obj.my_param_string) # Should be "builtins.str" + reveal_type(obj.my_pyram_string_none) # Should be "typing.Optional[str]" + reveal_type(obj.my_param_string_no_default) # Should be "typing.Optional[str]" + + reveal_type(obj.my_test_string) # Should be "builtins.str" + reveal_type(obj.my_number) # Should be "builtins.float" + reveal_type(obj.my_value_annotated) # Should be "builtins.int" + +if __name__ == "__main__": + test_myclass() \ No newline at end of file diff --git a/static-typing/problem.md b/static-typing/problem.md new file mode 100644 index 000000000..390787f0e --- /dev/null +++ b/static-typing/problem.md @@ -0,0 +1,52 @@ +# Param - Static Typing - Problem + +[HoloViz Param](https://param.holoviz.org/) is a powerful library for defining parameters on `param.Parameterized` classes: + +```python +import param + +class MyClass(param.Parameterized): + my_parameter = param.String(default="some value", doc="A custom parameter") +``` + +## The Core Problem + +**Param does not play well with Python's static typing ecosystem.** + +Modern Python development expects type annotations to: +- Enable code completion and inline documentation in editors (like VS Code, PyCharm) +- Allow static analysis and error checking with tools like mypy and pylint +- Make code easier to understand, refactor, and maintain + +But with Param, these benefits are lost: + +- **No Automatic Type Inference:** When you declare a parameter (e.g. `my_parameter = param.String(...)`), neither the class nor its instances have a type annotation for `my_parameter`. Editors and type checkers cannot know its type. +- **Poor Editor Experience:** You don't get tab completion, hover help, or docstrings for parameter attributes on instances. For example, `instance.my_parameter` is invisible to your IDE. +- **False Type Checker Warnings:** Type checkers like mypy and pylint may complain about missing attributes or unknown types, even when the code is correct. For example, `instance.data.iloc` may trigger errors if `data` is a `param.DataFrame` parameter. +- **Unhelpful Constructor Signatures:** The `__init__` method of a `param.Parameterized` class does not show which parameters can be set, making it hard for users and tools to know what arguments are accepted. + +### Why This Is a Big Deal + +- **Static typing is now standard in Python.** Most modern libraries, editors, and teams expect it. +- **Without static typing, Param is harder to use, debug, and maintain.** +- **New users struggle to discover and use parameters.** +- **Advanced tooling (refactoring, auto-completion, documentation) is much less effective.** + +## Example + +```python +class Scatter2dWithSelectionComponent(param.Parameterized): + data: pd.DataFrame = param.DataFrame(precedence=0.1) +``` + +Even with a type annotation, editors and type checkers do not recognize `data` as a `pd.DataFrame` on the instance. This leads to poor auto-completion and false warnings. + +## What Is Needed + +- **Automatic type annotations for parameters.** Parameter attributes should be typed so editors and type checkers know their types. +- **Constructor signatures that reflect available parameters.** +- **Better editor integration for auto-completion and documentation.** +- **No breaking changes unless absolutely necessary.** +- **Bonus:** If the solution speeds up Param or makes it compatible with dataclasses or Pydantic, that's a huge win. + +See the full discussion and examples in [GitHub Issue #376](https://github.com/holoviz/param/issues/376). \ No newline at end of file