Merge pull request numba#8795 from stuartarchibald/wip/pending_deprec…

…ation_pycc

Add pending-deprecation warnings for ``numba.pycc``

docs/source/reference/aot-compilation.rst

@@ -3,6 +3,9 @@
 Ahead-of-Time compilation
 =========================
 
+.. note:: This module is pending deprecation. Please see
+          :ref:`deprecation-numba-pycc` for more information.
+
 .. currentmodule:: numba.pycc
 
 .. class:: CC(extension_name, source_module=None)

docs/source/reference/deprecation.rst

@@ -387,6 +387,71 @@ for example::
   print(foo("a string"))
 
 
+.. _deprecation-numba-pycc:
+
+Deprecation of the ``numba.pycc`` module
+========================================
+Numba has supported some degree of Ahead-of-Time (AOT) compilation through the
+use of the tools in the ``numba.pycc`` module. This capability is very important
+to the Numba project and following an assessment of the viability of the current
+approach, it was decided to deprecate it in favour of developing new technology
+to better meet current needs.
+
+Reason for deprecation
+----------------------
+
+There are a number of reasons for this deprecation.
+
+* ``numba.pycc`` tools create C-Extensions that have symbols that are only
+  usable from the Python interpreter, they are not compatible with calls made
+  from within code compiled using Numba's JIT compiler. This drastically reduces
+  the utility of AOT compiled functions.
+* ``numba.pycc`` has some reliance on ``setuptools`` (and ``distutils``) which
+  is something Numba is trying to reduce, particularly due to the upcoming
+  removal of ``distutils`` in Python 3.12.
+* The ``numba.pycc`` compilation chain is very limited in terms of its feature
+  set in comparison to Numba's JIT compiler, it also has numerous technical
+  issues to do with declaring and linking both internal and external libraries.
+* The number of users of ``numba.pycc`` is assumed to be quite small, this was
+  indicated through discussions at a Numba public meeting on 2022-10-04 and
+  issue #8509.
+* The Numba project is working on new innovations in the AOT compiler space and
+  the maintainers consider it a better use of resources to develop these than
+  maintain and develop ``numba.pycc``.
+
+Example(s) of the impact
+------------------------
+
+Any source code using ``numba.pycc`` would fail to work once the functionality
+has been removed.
+
+Schedule
+--------
+
+This feature will be removed with respect to this schedule:
+
+* Pending-deprecation warnings will be issued in version 0.57.0.
+* Deprecation warnings will be issued once a replacement is developed.
+* Deprecation warnings will be given for a minimum of two releases prior to full
+  removal.
+
+Recommendations
+---------------
+
+Projects that need/rely on the deprecated behaviour should pin their dependency
+on Numba to a version prior to removal of this behaviour, or consider following
+replacement instructions below that outline how to adjust to the change.
+
+Replacement
+-----------
+
+A replacement for this functionality is being developed as part of the Numba
+2023 development focus. The ``numba.pycc`` module will not be removed until this
+replacement functionality is able to provide similar utility and offer an
+upgrade path. At the point of the new technology being deemed suitable,
+replacement instructions will be issued.
+
+
 Deprecation of eager compilation of CUDA device functions
 =========================================================
 

docs/source/user/pycc.rst

@@ -8,6 +8,9 @@ Compiling code ahead of time
 While Numba's main use case is :term:`Just-in-Time compilation`, it also
 provides a facility for :term:`Ahead-of-Time compilation` (AOT).
 
+.. note:: This module is pending deprecation. Please see
+          :ref:`deprecation-numba-pycc` for more information.
+
 
 Overview
 ========

numba/pycc/__init__.py

@@ -1,5 +1,23 @@
 # -*- coding: utf-8 -*-
 
+import warnings
+from numba.core.errors import NumbaPendingDeprecationWarning
+
 # Public API
 from .cc import CC
 from .decorators import export, exportmany
+
+# If use of anything is attempted through the `pycc` import path this warning
+# will be shown.
+__pycc_deprecation_doc_url = ("https://numba.readthedocs.io/en/stable/"
+                              "reference/deprecation.html"
+                              "#deprecation-of-the-numba-pycc-module")
+__pycc_pending_deprecation_message = ("The 'pycc' module is pending "
+                                      "deprecation. Replacement technology is "
+                                      "being developed.\n\n"
+                                      "Pending Deprecation in Numba 0.57.0. "
+                                      "For more information please see: "
+                                      f"{__pycc_deprecation_doc_url}")
+
+_pend_dep = NumbaPendingDeprecationWarning(__pycc_pending_deprecation_message)
+warnings.warn(_pend_dep, stacklevel=2)

numba/tests/support.py

@@ -50,7 +50,6 @@
     models,
 )
 from numba.core.datamodel.models import OpaqueModel
-from numba.pycc.platform import external_compiler_works
 
 try:
     import scipy
@@ -672,6 +671,9 @@ def skip_if_no_external_compiler(self):
         decorator so as to make it "lazy" via runtime evaluation opposed to
         running at test-discovery time.
         """
+        # This is a local import to avoid deprecation warnings being generated
+        # through the use of the numba.pycc module.
+        from numba.pycc.platform import external_compiler_works
         if not external_compiler_works():
             self.skipTest("No suitable external compiler was found.")
 

numba/tests/test_deprecations.py

@@ -136,6 +136,30 @@ def foo():
             self.check_warning(w, "numba.generated_jit is deprecated",
                                NumbaDeprecationWarning)
 
+    @TestCase.run_test_in_subprocess
+    def test_pycc_module(self):
+        # checks import of module warns
+
+        with warnings.catch_warnings(record=True) as w:
+            warnings.simplefilter("always",
+                                  category=NumbaPendingDeprecationWarning)
+            import numba.pycc # noqa: F401
+
+            expected_str = ("The 'pycc' module is pending deprecation.")
+            self.check_warning(w, expected_str, NumbaPendingDeprecationWarning)
+
+    @TestCase.run_test_in_subprocess
+    def test_pycc_CC(self):
+        # check the most commonly used functionality (CC) warns
+
+        with warnings.catch_warnings(record=True) as w:
+            warnings.simplefilter("always",
+                                  category=NumbaPendingDeprecationWarning)
+            from numba.pycc import CC # noqa: F401
+
+            expected_str = ("The 'pycc' module is pending deprecation.")
+            self.check_warning(w, expected_str, NumbaPendingDeprecationWarning)
+
 
 if __name__ == '__main__':
     unittest.main()

numba/tests/test_pycc.py

@@ -13,10 +13,6 @@
 import llvmlite.binding as ll
 
 from numba.core import utils
-from numba.pycc.decorators import clear_export_registry
-from numba.pycc.platform import find_shared_ending, find_pyext_ending
-from numba.pycc.platform import external_compiler_works
-
 from numba.tests.support import (TestCase, tag, import_dynamic, temp_directory,
                                  has_blas)
 import unittest
@@ -50,6 +46,10 @@ def test_windows_compiler_validity(self):
         # When inside conda-build VSINSTALLDIR should be set and windows should
         # have a valid compiler available, `external_compiler_works()` should
         # agree with this. If this is not the case then error out to alert devs.
+
+        # This is a local import to avoid deprecation warnings being generated
+        # through the use of the numba.pycc module.
+        from numba.pycc.platform import external_compiler_works
         is_running_conda_build = os.environ.get('CONDA_BUILD', None) is not None
         if is_running_conda_build:
             if os.environ.get('VSINSTALLDIR', None) is not None:
@@ -71,6 +71,10 @@ def tearDown(self):
         # Since we're executing the module-under-test several times
         # from the same process, we must clear the exports registry
         # between invocations.
+
+        # This is a local import to avoid deprecation warnings being generated
+        # through the use of the numba.pycc module.
+        from numba.pycc.decorators import clear_export_registry
         clear_export_registry()
 
     @contextlib.contextmanager
@@ -136,6 +140,9 @@ def test_cc_properties(self):
         self.assertTrue(os.path.basename(f).startswith('pycc_test_simple.'), f)
         if sys.platform.startswith('linux'):
             self.assertTrue(f.endswith('.so'), f)
+            # This is a local import to avoid deprecation warnings being
+            # generated through the use of the numba.pycc module.
+            from numba.pycc.platform import find_pyext_ending
             self.assertIn(find_pyext_ending(), f)
 
     def test_compile(self):