gh-128384: Use a context variable for warnings.catch_warnings

[nascheme]

Make warnings.catch_warnings() use a context variable for holding the warning filtering state if the sys.flags.context_aware_warnings flag is set to true. This makes using the context manager thread-safe in multi-threaded programs and safe for asyncio coroutines and tasks. The flag is true by default in free-threaded builds and is otherwise false. The value of the flag can be overridden by the the -X thread_safe_warnings command-line option or by the PYTHON_CONTEXT_AWARE_WARNINGS environment variable.

It is expected that one day the flag might default to true for all Python builds, not just the free-threaded one. However, I think it makes sense to not commit to a schedule to do that until we have a better idea of what code is affected by this change. Feedback from people using the free-threaded build should provide guidance.

This PR is on top of gh-128209.

A previous version of this PR used a single flag to control both behavior of Thread inheriting the context and also the catch_warnings context manager. However, based on some feedback, I've decided that two flags makes things more clear. Likely programs would like to set both flags to true to get the most intuitive behavior.

When the context_aware_warnings flag is true and a catch_warnings() context is active, added filters go into a list of filters stored in the contextvar, not the warnings.filters list. The filters in warnings.filters are still applied but they apply after the contextvar filters.

That difference with how warnings.filters works is probably the most likely thing to cause broken user code. In the unit tests, I had to change a number of references to warnings.filters to warnings._get_filters(). That function returns the list of filters from the current context or the global filters if there is no context active. Perhaps _get_filters() should become a public function? I think it would be better if examining and manipulating that list was discouraged and so that's why I made the function non-public, at least for now.

I created _py_warnings.py to contain the Python implementation for the warnings module. This matches other modules like decimal that have Python and C versions of the same module. In the case of warnings things are a bit more complicated since you can assign to module globals (the filters list or the showwarning() function are common to override). This is a cleaner organization, IMHO. Now warnings.py just imports the parts it needs and the unit tests are a bit simpler.

I cleaned up the unit tests for warnings a bit. Passing module to catch_warnings() actually serves no purpose. Since sys.modules['warnings'] is already swapped to the module under test, the code does the right thing even when that parameter is not given. I also replaced calls to original_warnings.catch_warnings() with self.module.catch_warnings(). Both those calls do the same thing but the second spelling seems less confusing to me.

• Issue: warnings.catch_warnings is not thread safe or async safe #128384

pyperformance comparison

---

📚 Documentation preview 📚: https://cpython-previews--130010.org.readthedocs.build/

[kumaraditya303]

I think we should add some tests which use both asyncio and warnings to check for correct contexts in async environments too.

[hugovk]

@nascheme @kumaraditya303 Please could you check the buildbots? https://buildbot.python.org/#/release_status

AMD64 CentOS9 NoGIL Refleaks 3.x (tier-1: breaks need immediate fix/revert) and ARM64 MacOS M1 Refleaks NoGIL 3.x (tier-2: breaks need fix/revert within 24h) have been failing for 6 days, and this PR seems a likely cause?

For example:

test_threaded_context (test.test_warnings.CThreadTests.test_threaded_context) ... Warning -- Uncaught thread exception: TypeErrorException in thread Thread-5 (run_a):
Traceback (most recent call last):
  File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/threading.py", line 1079, in _bootstrap_inner
    self._context.run(self.run)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/threading.py", line 1021, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/test_warnings/__init__.py", line 1650, in run_a
    self.module.warn('run_a warning', UserWarning)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/_py_warnings.py", line 227, in _showwarnmsg
    raise TypeError("warnings.showwarning() must be set to a "
                    "function or method")TypeError: warnings.showwarning() must be set to a function or method

https://buildbot.python.org/#/builders/1368/builds/2970/steps/6/logs/stdio

[nascheme]

I'll look into it. My guess is it is a flaky test and the test could just be disabled as a stop-gap.

[hugovk]

Thanks, here's a draft revert PR has passing buildbots: #132601.

[nascheme]

I believe this should fix the test failure without reverting the whole change: GH-132611. I'm waiting for CI tests to complete.