docs about LazyConfig

Summary:
fix facebookresearch#3225

Pull Request resolved: fairinternal/detectron2#553

Reviewed By: wat3rBro

Differential Revision: D29731838

Pulled By: ppwwyyxx

fbshipit-source-id: 69ed27681747cf00a2fa682c245369c3d1ee8945

MODEL_ZOO.md

@@ -11,8 +11,8 @@ You can access these models from code using [detectron2.model_zoo](https://detec
 In addition to these official baseline models, you can find more models in [projects/](projects/).
 
 #### How to Read the Tables
-* The "Name" column contains a link to the config file. Running `tools/train_net.py --num-gpus 8` with this config file
-  will reproduce the model.
+* The "Name" column contains a link to the config file. Models can be reproduced using `tools/train_net.py` with the corresponding yaml config file,
+  or `tools/lazyconfig_train_net.py` for python config files.
 * Training speed is averaged across the entire training.
   We keep updating the speed with latest version of detectron2/pytorch/etc.,
   so they might be different from the `metrics` file.

detectron2/config/lazy.py

@@ -229,6 +229,11 @@ def load(filename: str, keys: Union[None, str, Tuple[str, ...]] = None):
     @staticmethod
     def save(cfg, filename: str):
         """
+        Save a config object to a yaml file.
+        Note that when the config dictionary contains complex objects (e.g. lambda),
+        it can't be saved to yaml. In that case we will print an error and
+        attempt to save to a pkl file instead.
+
         Args:
             cfg: an omegaconf config object
             filename: yaml file name to save the config file
@@ -311,7 +316,10 @@ def safe_update(cfg, key, value):
     @staticmethod
     def to_py(cfg, prefix: str = "cfg."):
         """
-        Convert a config object into its equivalent Python code.
+        Try to convert a config object into its equivalent Python code.
+
+        Note that this is not always possible. So the returned results are mainly
+        meant to be human-readable, and not meant to be loaded back.
 
         Args:
             cfg: an omegaconf config object

detectron2/engine/defaults.py

@@ -132,9 +132,11 @@ def default_argument_parser(epilog=None):
     )
     parser.add_argument(
         "opts",
-        help="Modify config options by adding 'KEY VALUE' pairs at the end of the command. "
-        "See config references at "
-        "https://detectron2.readthedocs.io/modules/config.html#config-references",
+        help="""
+Modify config options at the end of the command. For Yacs configs, use
+space-separated "PATH.KEY VALUE" pairs.
+For python-based LazyConfig, use "path.key=value".
+        """.strip(),
         default=None,
         nargs=argparse.REMAINDER,
     )

detectron2/utils/analysis.py

@@ -13,6 +13,7 @@
     "flop_count_operators",
     "parameter_count_table",
     "parameter_count",
+    "FlopCountAnalysis",
 ]
 
 FLOPS_MODE = "flops"

docs/modules/config.rst

@@ -1,4 +1,4 @@
-detectron2.config 
+detectron2.config
 =========================
 
 Related tutorials: :doc:`../tutorials/configs`, :doc:`../tutorials/extend`.
@@ -10,7 +10,7 @@ Related tutorials: :doc:`../tutorials/configs`, :doc:`../tutorials/extend`.
     :inherited-members:
 
 
-Config References
+Yaml Config References
 -----------------
 
 .. literalinclude:: ../../detectron2/config/defaults.py

docs/tutorials/configs.md

@@ -1,24 +1,17 @@
-# Configs
+# Yacs Configs
 
 Detectron2 provides a key-value based config system that can be
 used to obtain standard, common behaviors.
 
-Detectron2's config system uses YAML and [yacs](https://github.com/rbgirshick/yacs).
-In addition to the [basic operations](../modules/config.html#detectron2.config.CfgNode)
-that access and update a config, we provide the following extra functionalities:
-
-1. The config can have `_BASE_: base.yaml` field, which will load a base config first.
-   Values in the base config will be overwritten in sub-configs, if there are any conflicts.
-   We provided several base configs for standard model architectures.
-2. We provide config versioning, for backward compatibility.
-   If your config file is versioned with a config line like `VERSION: 2`,
-   detectron2 will still recognize it even if we change some keys in the future.
-
-Config file is a very limited language.
-We do not expect all features in detectron2 to be available through configs.
+This system uses YAML and [yacs](https://github.com/rbgirshick/yacs).
+Yaml is a very limited language,
+so we do not expect all features in detectron2 to be available through configs.
 If you need something that's not available in the config space,
 please write code using detectron2's API.
 
+With the introduction of a more powerful [LazyConfig system](lazyconfigs.md),
+we no longer add functionality / new keys to the Yacs/Yaml-based config system.
+
 ### Basic Usage
 
 Some basic usage of the `CfgNode` object is shown here. See more in [documentation](../modules/config.html#detectron2.config.CfgNode).
@@ -32,6 +25,11 @@ cfg.merge_from_list(["MODEL.WEIGHTS", "weights.pth"])   # can also load values f
 print(cfg.dump())  # print formatted configs
 ```
 
+In addition to the basic Yaml syntax, the config file can
+define a `_BASE_: base.yaml` field, which will load a base config file first.
+Values in the base config will be overwritten in sub-configs, if there are any conflicts.
+We provided several base configs for standard model architectures.
+
 Many builtin tools in detectron2 accept command line config overwrite:
 Key-value pairs provided in the command line will overwrite the existing values in the config file.
 For example, [demo.py](../../demo/demo.py) can be used with
@@ -43,7 +41,6 @@ For example, [demo.py](../../demo/demo.py) can be used with
 To see a list of available configs in detectron2 and what they mean,
 check [Config References](../modules/config.html#config-references)
 
-
 ### Configs in Projects
 
 A project that lives outside the detectron2 library may define its own configs, which will need to be added
@@ -61,9 +58,3 @@ add_pointrend_config(cfg)  # add pointrend's default config
    to share common parts between configs.
 
 2. Keep the configs you write simple: don't include keys that do not affect the experimental setting.
-
-3. Keep a version number in your configs (or the base config), e.g., `VERSION: 2`,
-   for backward compatibility.
-	 We print a warning when reading a config without version number.
-   The official configs do not include version number because they are meant to
-   be always up-to-date.

docs/tutorials/extend.md

@@ -15,14 +15,17 @@ which is a challenge for any research engineering project of a significant size:
 In detectron2, there are two types of interfaces that address this tension together:
 
 1. Functions and classes that take a config (`cfg`) argument
+   created from a yaml file
    (sometimes with few extra arguments).
 
    Such functions and classes implement
-   the "standard default" behavior: it will read what it needs from the
+   the "standard default" behavior: it will read what it needs from a given
    config and do the "standard" thing.
-   Users only need to load a given config and pass it around, without having to worry about
+   Users only need to load an expert-made config and pass it around, without having to worry about
    which arguments are used and what they all mean.
 
+   See [Yacs Configs](configs.md) for a detailed tutorial.
+
 2. Functions and classes that have well-defined explicit arguments.
 
    Each of these is a small building block of the entire system.
@@ -33,6 +36,8 @@ In detectron2, there are two types of interfaces that address this tension toget
    When you need to implement something not supported by the "standard defaults"
    included in detectron2, these well-defined components can be reused.
 
+   The [LazyConfig system](lazyconfigs.md) relies on such functions and classes.
+
 3. A few functions and classes are implemented with the
    [@configurable](../modules/config.html#detectron2.config.configurable)
    decorator - they can be called with either a config, or with explicit arguments, or a mixture of both.
@@ -42,7 +47,7 @@ In detectron2, there are two types of interfaces that address this tension toget
 
    1. Config-only:
       ```python
-      # load proper config file, then
+      # load proper yaml config file, then
       model = build_model(cfg)
       ```
 

docs/tutorials/index.rst

@@ -16,4 +16,5 @@ Tutorials
    training
    evaluation
    configs
+   lazyconfigs
    deployment

docs/tutorials/lazyconfigs.md

@@ -0,0 +1,158 @@
+# Lazy Configs
+
+The traditional yacs-based config system provides basic, standard functionalities.
+However, it does not offer enough flexibility for many new projects.
+We develop an alternative, non-intrusive config system that can be used with
+detectron2 or potentially any other complex projects.
+
+## Python Syntax
+
+Our config objects are still dictionaries. Instead of using Yaml to define dictionaries,
+we create dictionaries in Python directly. This gives users the following power that
+doesn't exist in Yaml:
+
+* Easily manipulate the dictionary (addition & deletion) using Python.
+* Write simple arithmetics or call simple functions.
+* Use more data types / objects.
+* Import / compose other config files, using the familiar Python import syntax.
+
+A Python config file can be loaded like this:
+```python
+# config.py:
+a = dict(x=1, y=2, z=dict(xx=1))
+b = dict(x=3, y=4)
+
+# my_code.py:
+from detectron2.config import LazyConfig
+cfg = LazyConfig.load("path/to/config.py")  # an omegaconf dictionary
+assert cfg.a.z.xx == 1
+```
+
+After `LazyConfig.load`, `cfg` will be a dictionary that contains all dictionaries
+defined in the global scope of the config file. Note that:
+* All dictionaries are turned to an [omegaconf](https://omegaconf.readthedocs.io/)
+  config object during loading. This enables access to omegaconf features,
+  such as its [access syntax](https://omegaconf.readthedocs.io/en/2.1_branch/usage.html#access-and-manipulation)
+  and [interoplation](https://omegaconf.readthedocs.io/en/2.1_branch/usage.html#variable-interpolation).
+* Absolute imports in `config.py` works the same as in regular Python.
+* Relative imports can only import dictionaries from config files.
+  They are simply a syntax sugar for [LazyConfig.load_rel](../modules/config.html#detectron2.config.LazyConfig.load_rel).
+  They can load Python files at relative path without requiring `__init__.py`.
+
+## Recursive Instantiation
+
+The LazyConfig system heavily uses recursive instantiation, which is a pattern that
+uses a dictionary to describe a
+call to a function/class. The dictionary consists of:
+
+1. A "\_target\_" key which contains path to the callable, such as "module.submodule.class_name".
+2. Other keys that represent arguments to pass to the callable. Arguments themselves can be defined
+   using recursive instantiation.
+
+We provide a helper function [LazyCall](../modules/config.html#detectron2.config.LazyCall) that helps create such dictionaries.
+The following code using `LazyCall`
+```python
+from detectron2.config import LazyCall as L
+from my_app import Trainer, Optimizer
+cfg = L(Trainer)(
+  optimizer=L(Optimizer)(
+    lr=0.01,
+    algo="SGD"
+  )
+)
+```
+creates a dictionary like this:
+```
+cfg = {
+  "_target_": "my_app.Trainer",
+  "optimizer": {
+    "_target_": "my_app.Optimizer",
+    "lr": 0.01, "algo": "SGD"
+  }
+}
+```
+
+By representing objects using such dictionaries, a general
+[instantiate](../modules/config.html#detectron2.config.instantiate)
+function can turn them into actual objects, i.e.:
+```python
+from detectron2.config import instantiate
+trainer = instantiate(cfg)
+# equivalent to:
+# from my_app import Trainer, Optimizer
+# trainer = Trainer(optimizer=Optimizer(lr=0.01, algo="SGD"))
+```
+
+This pattern is powerful enough to describe very complex objects, e.g.:
+
+ <details>
+ <summary>
+A Full Mask R-CNN described in recursive instantiation (click to expand)
+ </summary>
+
+```eval_rst
+.. literalinclude:: ../../configs/common/models/mask_rcnn_fpn.py
+  :language: python
+  :linenos:
+```
+
+ </details>
+
+There are also objects or logic that cannot be described simply by a dictionary,
+such as reused objects or method calls. They may require some refactoring
+to work with recursive instantiation.
+
+## Using Model Zoo LazyConfigs
+
+We provide some configs in the model zoo using the LazyConfig system, for example:
+
+* [common baselines](../../configs/common/).
+* [new Mask R-CNN baselines](../../configs/new_baselines/)
+
+After installing detectron2, they can be loaded by the model zoo API
+[model_zoo.get_config](../modules/model_zoo.html#detectron2.model_zoo.get_config).
+
+Our model zoo configs follow some simple conventions, e.g.
+`cfg.model` defines a model object, `cfg.dataloader.{train,test}` defines dataloader objects,
+and `cfg.train` contains training options in key-value form.
+We provide a reference training script
+[tools/lazyconfig_train_net.py](../../tools/lazyconfig_train_net.py),
+that can train/eval our model zoo configs.
+It also shows how to support command line value overrides.
+
+Nevertheless, you are free to define any custom structure for your project and use it
+with your own scripts.
+
+To demonstrate the power and flexibility of the new system, we show that
+[a simple config file](../../configs/Misc/torchvision_imagenet_R_50.py)
+can let detectron2 train an ImageNet classification model from torchvision, even though
+detectron2 contains no features about ImageNet classification.
+This can serve as a reference for using detectron2 in other deep learning tasks.
+
+## Summary
+
+By using recursive instantiation to create objects,
+we avoid passing a giant config to many places, because `cfg` is only passed to `instantiate`.
+This has the following benefits:
+
+* It's __non-intrusive__: objects to be constructed are config-agnostic, regular Python
+  functions/classes.
+  They can even live in other libraries. For example,
+  `{"_target_": "torch.nn.Conv2d", "in_channels": 10, "out_channels": 10, "kernel_size": 1}`
+  defines a conv layer.
+* __Clarity__ of what function/classes will be called, and what arguments they use.
+* `cfg` doesn't need pre-defined keys and structures. It's valid as long as it translates to valid
+  code. This gives a lot more __flexibility__.
+* You can still pass huge dictionaries as arguments, just like the old way.
+
+Putting recursive instantiation together with the Python config file syntax, the config file
+looks a lot like the code that will be executed:
+
+![img](./lazyconfig.jpg)
+
+However, the config file just defines dictionaries, which can be easily manipulated further
+by composition or overrides.
+The corresponding code will only be executed
+later when `instantiate` is called. In some way,
+in config files we're writing "editable code" that will be "lazily executed" later when needed.
+That's why we call this system "LazyConfig".

docs/tutorials/models.md

@@ -1,6 +1,8 @@
 # Use Models
 
-Models (and their sub-models) in detectron2 are built by
+## Build Models from Yacs Config
+From a yacs config object,
+models (and their sub-models) can be built by
 functions such as `build_model`, `build_backbone`, `build_roi_heads`:
 ```python
 from detectron2.modeling import build_model

docs/tutorials/training.md

@@ -22,7 +22,8 @@ It includes the following two instantiations:
   provides a minimal training loop for single-cost single-optimizer single-data-source training, with nothing else.
   Other tasks (checkpointing, logging, etc) can be implemented using
   [the hook system](../modules/engine.html#detectron2.engine.HookBase).
-* [DefaultTrainer](../modules/engine.html#detectron2.engine.defaults.DefaultTrainer) is a `SimpleTrainer` initialized from a config, used by
+* [DefaultTrainer](../modules/engine.html#detectron2.engine.defaults.DefaultTrainer) is a `SimpleTrainer` initialized from a
+  yacs config, used by
   [tools/train_net.py](../../tools/train_net.py) and many scripts.
   It includes more standard default behaviors that one might want to opt in,
   including default configurations for optimizer, learning rate schedule,