Skip to content

[ty] Tell the user why we inferred the Python version we inferred #18082

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

AlexWaygood
Copy link
Member

Summary

I'm blaming @MichaReiser for this. He said it would be easy in #18068 (comment) and... it wasn't 😆

This PR propagates the origin of the inferred Python version (CLI flag, pyproject.toml, ty.toml, etc.) from the ty_project crate to the ty_python_semantic crate. (Specifically, a new python_version_source field is added to the ty_python_semantic::program::Program struct.) Having the origin of the Python version available allows us to add subdiagnostics that annotate the specific part of the config file that led us to infer a certain Python version.

A screenshot to demonstrate this in action:

image

Test Plan

I added integration tests to the ty crate.

@AlexWaygood AlexWaygood added ty Multi-file analysis & type inference diagnostics Related to reporting of diagnostics. labels May 14, 2025
@AlexWaygood AlexWaygood force-pushed the alex/tell-me-why-py-version branch 2 times, most recently from 0c61c94 to 58e0a85 Compare May 14, 2025 01:29
Copy link
Contributor

github-actions bot commented May 14, 2025

mypy_primer results

No ecosystem changes detected ✅

@AlexWaygood AlexWaygood force-pushed the alex/tell-me-why-py-version branch from 58e0a85 to e10016c Compare May 14, 2025 01:37
Copy link
Contributor

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Copy link
Member

@MichaReiser MichaReiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great. Unfortunately, I expect this to break persistent caching where we'd use a different persistent cache for different python versions.

For now, I think the best we can do, without breaking persistent caching), is to say if the value comes from the CLI or configuration but without saying exactly from which configuration.

@@ -239,8 +239,8 @@ impl ProjectMetadata {
&self.extra_configuration_paths
}

pub fn to_program_settings(&self, system: &dyn System) -> ProgramSettings {
self.options.to_program_settings(self.root(), system)
pub fn to_program_settings(&self, db: &dyn crate::Db) -> ProgramSettings {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry, but we can't use db here. It will break our approach to persistent caching.

What's important for persistent caching is that we can resolve the ProgramSettings before constructing the db. We then hash the ProgramSettings and use the hash to lookup the cache file and load the database from it. This allows us to support different persistent caches when running ty with different python versions

Comment on lines +136 to +143
impl<T> RangedValue<T>
where
T: Copy,
{
pub fn inner_copied(&self) -> T {
self.value
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't the call site write value = *ranged instead?

Comment on lines +18 to +19
#[returns(ref)]
pub python_version_source: ValueSource,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to wrap PythonVersion in a struct that stores both the source and the version to reduce the top-level fields and make it harder to only update python_version but forget to update the source (which you didn't!)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this also mean a program with the same Python version but defined at separate source would invalidate the entire program? This isn't that important issue as changing the source of a Python version is not going to be that frequent in practice (I think) and the benefit that it provides is more useful IMO.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could ignore the source when computing the cache key

@@ -59,7 +59,7 @@ impl ProjectDatabase {
// we may want to have a dedicated method for this?

// Initialize the `Program` singleton
let program_settings = project_metadata.to_program_settings(db.system());
let program_settings = project_metadata.to_program_settings(&db);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the comment above about persistent caching (refer to my other inline comment)

@@ -17,7 +17,7 @@ pub mod pyproject;
pub mod settings;
pub mod value;

#[derive(Debug, PartialEq, Eq)]
#[derive(Debug, PartialEq, Eq, Clone)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer if this doesn't have to be clone. It's a very heavy data structure. It's not clear to me where we have to clone it.

Suggested change
#[derive(Debug, PartialEq, Eq, Clone)]
#[derive(Debug, PartialEq, Eq)]

@sharkdp sharkdp removed their request for review May 14, 2025 06:35
@carljm carljm removed their request for review May 15, 2025 02:53
@MichaReiser
Copy link
Member

Another way to work around the no db problem is to store the SystemPath in the python-version's RangedValue and lazily converting the path to a file when emitting a diagnostic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
diagnostics Related to reporting of diagnostics. ty Multi-file analysis & type inference
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants