Core API Details

Structure:

1) Algorithms

• An algorithm must be written as a class within a genrl/<algo_domain><domain>/<algo_name>/<algo_name>.py file
• Must give option for policy, value function (according to policies.py and values.py) and envs.
• The algo class should have at least the following 4 methods, __init__(), select_action(), update_params() and learn(). Apart from these, helper functions can be added.
• __init__():
  • conversion of user class arguments to class variables (self.x)
  • instantiating logging and seeds.
  • create model
• select_action():
  • implement policy using classes available in the policies.py file
  • return selected action
• update_params():
  • compute loss
  • take learning step
• learn():
  • implements training loop for algorithm
  • should include rendering, printing and tensorboard logging options.
• Add inline documentation (Refer to implemented classes)

2) Policies and Value functions

• Includes classes for common policy and value functions
• Any newly implemented policy or value function class should be implemented and recorded in the policy_registry or value_registry to enable access from get_policy_from_name() and get_value_from_name()
• A policy class should implement separate forward() and sample_action() methods since they may be called separately from the algo class. [This may change following the PR for GeneralPolicy]
• Don't forget to account for all possible env and state/action space configurations.
• Add inline documentation (Refer to implemented classes)

Note:: Any implemented method or class must be added (if required) to the corresponding __init__.py file.

Linting and Style:

• We are following the pylint linter specifications.
• In order to ensure uniformity in terms of style, reformat your code using black before merging.

Contributing:

• Before working on an issue, comment on the issue that you'd like to work on it, so that one of the maintainers can assign it to you.
• Fork the main repository.
• For each issue, work on a new branch to your fork. Once you have committed to it, open a PR or draft PR to this repository.