Skip to content

Commit ff90402

Browse files
committed
Create RFC for a logging system and list some required features
1 parent b379b19 commit ff90402

File tree

1 file changed

+103
-0
lines changed

1 file changed

+103
-0
lines changed

RFC-0026-logging-system.md

+103
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
# PyTorch Logging System
2+
3+
## **Summary**
4+
Create a message logging system for PyTorch with the following requirements:
5+
6+
* All errors, warnings, and other messages generated by PyTorch should be
7+
emitted using the the logging system API
8+
9+
* The APIs for emitting messages and changing settings should all be consistent
10+
between C++ and Python
11+
12+
* Offer different message severity levels, including at least the following:
13+
14+
- **Info**: Emits a message without creating a warning or error. By default,
15+
this gets printed to stdout
16+
17+
- **Warning**: Emits a message as a warning. By default, this will turn into
18+
a Python warning
19+
20+
- **Error**: Emits a message as an error. By default, this will turn into
21+
a Python error
22+
23+
- TODO: Should we also have a **Fatal** severity for integration with
24+
Meta's internal logging system? A fatal message terminates the program
25+
26+
* Offer different classes of messages, including at least the following:
27+
28+
- **Default**: A catch-all message class
29+
30+
- **Nondeterministic**: Emitted when `torch.use_deterministic_algorithms(True)`
31+
is set and a nondeterministic operation is called
32+
33+
- **Deprecated**: Emitted when a deprecated function is called
34+
35+
- **Beta**: Emitted when a beta feature is called. See
36+
[PyTorch feature classifications](https://pytorch.org/blog/pytorch-feature-classification-changes/)
37+
38+
- **Prototype**: Emitted when a prototype feature is called. See
39+
[PyTorch feature classifications](https://pytorch.org/blog/pytorch-feature-classification-changes/)
40+
41+
* Continue using warning/error APIs that currently exist in PyTorch wherever
42+
possible. For instance, `TORCH_CHECK`, `TORCH_WARN`, and `TORCH_WARN_ONCE`
43+
should continue to be used in C++
44+
45+
- NOTE: These existing APIs don't currently have a concept of message classes,
46+
so that will need to be added
47+
48+
* Creating new message classes and severity levels should be easy
49+
50+
* Settings to turn warnings for a specific message class into errors
51+
52+
* Settings to disable specific message classes and severity levels
53+
54+
- TODO: However, most errors should not be disableable, right? Perhaps only
55+
some message classes should allow disabling or downgrading errors. For
56+
instance, currently in PyTorch, we can downgrade a nondeterministic error
57+
to a warning, but we wouldn't want to downgrade an error from invalid
58+
arguments given to an operation.
59+
60+
* Settings to avoid emitting duplicate messages generated by multiple
61+
`torch.distribted` ranks (related to issue
62+
[#68768](https://github.com/pytorch/pytorch/issues/68768))
63+
64+
* Ability to make a particular warning only warn once
65+
66+
- NOTE: Currently `TORCH_WARN_ONCE` does this in C++, but there is no Python
67+
equivalent
68+
69+
- TODO: Should there be a setting to turn a warn-always into a warn-once for
70+
a given message class and vice versa?
71+
72+
- TODO: Should warn-once be its own separate severity level?
73+
74+
* Settings can be changed from Python, C++, or environment variables
75+
76+
- Filtering warnings with Python command line arguments should
77+
remain possible. For instance, the following turns a `DeprecationWarning`
78+
into an error: `python -W error::DeprecationWarning your_script.py`
79+
80+
* Should integrate with Meta's internal logging system, which is
81+
[glog](https://github.com/google/glog)
82+
83+
- TODO: What are all the requirements that define "integrating with glog"
84+
85+
* Must be OSS-friendly, so it shouldn't require libraries (like glog) which may
86+
cause incompatibility issues for projects that use PyTorch
87+
88+
* TODO: Determine the requirements for the following concepts:
89+
90+
- Log files (default behavior and any settings)
91+
92+
93+
## **Motivation**
94+
Original issue: [link](https://github.com/pytorch/pytorch/issues/72948)
95+
96+
Currently, it is challenging for PyTorch developers to provide messages that
97+
act consistently between Python and C++.
98+
99+
It is also challenging for PyTorch users to manage the messages that PyTorch
100+
emits. For instance, if a PyTorch user happens to be calling PyTorch functions
101+
that emit lots of warnings, it can be difficult for them to filter out those
102+
warnings so that their project's users don't get bombarded with warnings that
103+
they don't need to see.

0 commit comments

Comments
 (0)