Check python version that was used to install pre-commit venvs (#43282)…

… (#43310)

Since we moved to Python 3.9 and started usign 3.9+ only features
line functools.cache, having Python 3.8 as default python version
will result in virtualenvs created with outdated python.

This pre-commit checks what is the global version of python3 and it
will complain if it is lower than 3.9. This is done via explicit
call of "python3", because this is what pre-commit does. Also
global setting is needed because otherwise new virtualenvs
created by pre-commit might be created using lower python version.

(cherry picked from commit 076da92)

.pre-commit-config.yaml

@@ -167,6 +167,12 @@ repos:
           \.cfg$|\.conf$|\.ini$|\.ldif$|\.properties$|\.readthedocs$|\.service$|\.tf$|Dockerfile.*$
   - repo: local
     hooks:
+      - id: check-min-python-version
+        name: Check minimum Python version
+        entry: ./scripts/ci/pre_commit/check_min_python_version.py
+        language: python
+        additional_dependencies: ['rich>=12.4.4']
+        require_serial: true
       - id: update-common-sql-api-stubs
         name: Check and update common.sql API stubs
         entry: ./scripts/ci/pre_commit/update_common_sql_api_stubs.py

contributing-docs/08_static_code_checks.rst

@@ -196,6 +196,8 @@ require Breeze Docker image to be built locally.
 +-----------------------------------------------------------+--------------------------------------------------------------+---------+
 | check-merge-conflict                                      | Check that merge conflicts are not being committed           |         |
 +-----------------------------------------------------------+--------------------------------------------------------------+---------+
+| check-min-python-version                                  | Check minimum Python version                                 |         |
++-----------------------------------------------------------+--------------------------------------------------------------+---------+
 | check-newsfragments-are-valid                             | Check newsfragments are valid                                |         |
 +-----------------------------------------------------------+--------------------------------------------------------------+---------+
 | check-no-airflow-deprecation-in-providers                 | Do not use DeprecationWarning in providers                   |         |

dev/breeze/doc/images/output_static-checks.txt

@@ -1 +1 @@
-97228aa2e9cdff69f86a9c0dfee5f83e
+a109ca358651bbfcf3a36d007fd0646a

dev/breeze/src/airflow_breeze/pre_commit_ids.py

@@ -63,6 +63,7 @@
     "check-lazy-logging",
     "check-links-to-example-dags-do-not-use-hardcoded-versions",
     "check-merge-conflict",
+    "check-min-python-version",
     "check-newsfragments-are-valid",
     "check-no-airflow-deprecation-in-providers",
     "check-no-providers-in-core-examples",

scripts/ci/pre_commit/check_min_python_version.py

@@ -0,0 +1,68 @@
+#!/usr/bin/env python
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from __future__ import annotations
+
+import subprocess
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).parent.resolve()))
+
+from common_precommit_utils import console
+
+# update this version when we switch to a newer version of Python
+required_version = tuple(map(int, "3.8".split(".")))
+required_version_str = f"{required_version[0]}.{required_version[1]}"
+global_version = tuple(
+    map(
+        int,
+        subprocess.run(
+            [
+                "python3",
+                "-c",
+                'import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}")',
+            ],
+            capture_output=True,
+            text=True,
+            check=True,
+        )
+        .stdout.strip()
+        .split("."),
+    )
+)
+
+
+if global_version < required_version:
+    console.print(f"[red]Python {required_version_str} or higher is required to install pre-commit.\n")
+    console.print(f"[green]Current version is {global_version}\n")
+    console.print(
+        "[bright_yellow]Please follow those steps:[/]\n\n"
+        f" * make sure that `python3 --version` is at least {required_version_str}\n"
+        f" * run `pre-commit clean`\n"
+        f" * run `pre-commit install --install-hooks`\n\n"
+    )
+    console.print(
+        "There are various ways you can set `python3` to point to a newer version of Python.\n\n"
+        f"For example run `pyenv global {required_version_str}` if you use pyenv, or\n"
+        f"you can use venv with python {required_version_str} when you use pre-commit, or\n"
+        "you can use `update-alternatives` if you use Ubuntu, or\n"
+        "you can set `PATH` to point to the newer version of Python.\n\n"
+    )
+    sys.exit(1)
+else:
+    console.print(f"Python version is sufficient: {required_version_str}")