ML-Agents Release 23

[4.0.0] - 2025-08-28

Major Changes

com.unity.ml-agents
Upgraded to Inference Engine 2.2.1 (#6212)
The minimum supported Unity version was updated to 6000.0. (#6207)
Merged the extension package com.unity.ml-agents.extensions to the main package com.unity.ml-agents. (#6227)

Minor Changes

com.unity.ml-agents
Removed broken sample from the package (#6230)
Moved to Unity Package documentation as the primary developer documentation. (#6232)
ml-agents / ml-agents-envs
Bumped grpcio version to >=1.11.0,<=1.53.2 (#6208)

ML-Agents Release 22

[3.0.0] - 2024-09-02

Major Changes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)
Upgraded to Sentis 2.0.0 (#6137)
Upgraded to Sentis 1.3.0-pre.3 (#6070)
Upgraded to Sentis 1.3.0-exp.2 (#6013)
The minimum supported Unity version was updated to 2023.2. (#6071)
ml-agents / ml-agents-envs
Upgraded to PyTorch 2.1.1. (#6013)

Minor Changes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)
Added no-graphics-monitor. (#6014)
ml-agents / ml-agents-envs
Update Installation.md (#6004)
Updated Using-Virtual-Environment.md (#6033)

Bug Fixes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)
Fix failing ci post upgrade (#6141)
Fixed missing assembly reference for google protobuf. (#6099)
Fixed missing tensor Dispose in ModelRunner. (#6028)
Fixed 3DBall sample package to remove Barracuda dependency. (#6030)
ml-agents / ml-agents-envs
Fix sample code indentation in migrating.md (#5840)
Fixed continuous integration tests (#6079)
Fixed bad like format (#6078)
Bumped numpy version to >=1.23.5,<1.24.0 (#6082)
Bumped onnx version to 1.15.0 (#6062)
Bumped protobuf version to >=3.6,<21 (#6062)

ML-Agents Release 21

[3.0.0-exp.1] - 2023-10-09

Major Changes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)

• Upgraded ML-Agents to Sentis 1.2.0-exp.2 and deprecated Barracuda. (#5979)
• The minimum supported Unity version was updated to 2022.3. (#5950)
• Added batched raycast sensor option. (#5950)

ml-agents / ml-agents-envs

• Updated to PyTorch 1.13.1 (#5982)
• Deprecated support for Python 3.8.x and 3.9.x (#5981)

Minor Changes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)

• Added DecisionStep parameter to DecisionRequester (#5940)
  • This will allow the staggering of execution timing when using multi-agents, leading to more stable performance.

ml-agents / ml-agents-envs

• Added timeout cli and yaml config file support for specifying environment timeout. (#5991)
• Added training config feature to evenly distribute checkpoints throughout training. (#5842)
• Updated training area replicator to add a condition to only replicate training areas when running a build. (#5842)

Bug Fixes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)

• Compiler errors when using IAsyncEnumerable with .NET Standard 2.1 enabled (#5951)

ml-agents / ml-agents-envs

ML-Agents Release 20

Package Versions

NOTE: It is strongly recommended that you use packages from the same release together for the best experience.

Package	Version
com.unity.ml-agents (C#)	v2.3.0-exp.3
com.unity.ml-agents.extensions (C#)	v0.6.1-preview
ml-agents (Python)	v0.30.0
ml-agents-envs (Python)	v0.30.0
gym-unity (Python)	v0.30.0
Communicator (C#/Python)	v1.5.0

Release Notes

Major Changes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)

• The minimum supported Unity version was updated to 2021.3. (#)

ml-agents / ml-agents-envs

• Add your trainers to the package using Ml-Agents Custom Trainers plugin. (#)
  • ML-Agents Custom Trainers plugin is an extensible plugin system to define new trainers based on the
    High level trainer API, read more here.
• Refactored core modules to make ML-Agents internal classes more generalizable to various RL algorithms. (#)
• The minimum supported Python version for ML-agents has changed to 3.8.13. (#)
• The minimum supported version of PyTorch was changed to 1.8.0. (#)
• Add shared critic configurability for PPO. (#)
• We moved UnityToGymWrapper and PettingZoo API to ml-agents-envs package. All these environments will be
  versioned under ml-agents-envs package in the future (#)

Minor Changes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)

• Added switch to RayPerceptionSensor to allow rays to be ordered left to right. (#)
  • Current alternating order is still the default but will be deprecated.
• Added suppport for enabling/disabling camera object attached to camera sensor in order to improve performance. (#)

ml-agents / ml-agents-envs

• Renaming the path that shadows torch with "mlagents/trainers/torch_entities" and update respective imports (#)

Bug Fixes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)

ml-agents / ml-agents-envs

ML-Agents Release 19

Package Versions

NOTE: It is strongly recommended that you use packages from the same release together for the best experience.

Package	Version
com.unity.ml-agents (C#)	v2.2.1-exp.1
com.unity.ml-agents.extensions (C#)	v0.6.1-preview
ml-agents (Python)	v0.28.0
ml-agents-envs (Python)	v0.28.0
gym-unity (Python)	v0.28.0
Communicator (C#/Python)	v1.5.0

Release Notes

Major Changes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)

• The minimum supported Unity version was updated to 2020.3. (#5673)
• Added a new feature to replicate training areas dynamically during runtime. (#5568)
• Update Barracuda to 2.3.1-preview (#5591)
• Update Input System to 1.3.0 (#5661)

ml-agents / ml-agents-envs / gym-unity (Python)

Minor Changes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)

• Added the capacity to initialize behaviors from any checkpoint and not just the latest one (#5525)
• Added the ability to get a read-only view of the stacked observations (#5523)

ml-agents / ml-agents-envs / gym-unity (Python)

• Set gym version in gym-unity to gym release 0.20.0 (#5540)
• Added support for having beta, epsilon, and learning rate on separate schedules (affects only PPO and POCA). (#5538)
• Changed default behavior to restart crashed Unity environments rather than exiting. (#5553)
  • Rate & lifetime limits on this are configurable via 3 new yaml options
    1. env_params.max_lifetime_restarts (--max-lifetime-restarts) [default=10]
    2. env_params.restarts_rate_limit_n (--restarts-rate-limit-n) [default=1]
    3. env_params.restarts_rate_limit_period_s (--restarts-rate-limit-period-s) [default=60]
• Deterministic action selection is now supported during training and inference(#5619)
  • Added a new --deterministic cli flag to deterministically select the most probable actions in policy. The same thing can
    be achieved by adding deterministic: true under network_settings of the run options configuration.(#5597)
  • Extra tensors are now serialized to support deterministic action selection in onnx. (#5593)
  • Support inference with deterministic action selection in editor (#5599)
• Added minimal analytics collection to LL-API (#5511)
• Update Colab notebooks for GridWorld example with DQN illustrating the use of the Python API and how to export to ONNX (#5643)

Bug Fixes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)

• Update gRPC native lib to universal for arm64 and x86_64. This change should enable ml-agents usage on mac M1 (#5283, #5519)
• Fixed a bug where ml-agents code wouldn't compile on platforms that didn't support analytics (PS4/5, XBoxOne) (#5628)

ml-agents / ml-agents-envs / gym-unity (Python)

• Fixed a bug where the critics were not being normalized during training. (#5595)
• Fixed the bug where curriculum learning would crash because of the incorrect run_options parsing. (#5586)
• Fixed a bug in multi-agent cooperative training where agents might not receive all of the states of
  terminated teammates. (#5441)
• Fixed wrong attribute name in argparser for torch device option (#5433)(#5467)
• Fixed conflicting CLI and yaml options regarding resume & initialize_from (#5495)
• Fixed failing tests for gym-unity due to gym 0.20.0 release (#5540)
• Fixed a bug in VAIL where the variational bottleneck was not properly passing gradients (#5546)
• Harden user PII protection logic and extend TrainingAnalytics to expose detailed configuration parameters. (#5512)

ML-Agents Release 18

Package Versions

NOTE: It is strongly recommended that you use packages from the same release together for the best experience.

Package	Version
com.unity.ml-agents (C#)	v2.1.0-exp.1
com.unity.ml-agents.extensions (C#)	v0.5.0-preview
ml-agents (Python)	v0.27.0
ml-agents-envs (Python)	v0.27.0
gym-unity (Python)	v0.27.0
Communicator (C#/Python)	v1.5.0

Release Notes

Minor Changes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)

• Updated Barracuda to 2.0.0-pre.3. (#5385)
• Fixed NullReferenceException when adding Behavior Parameters with no Agent. (#5382)
• Added stacking option in Editor for VectorSensorComponent. (#5376)

ml-agents / ml-agents-envs / gym-unity (Python)

• Locked cattrs dependency version to 1.6. (#5397)
• Added a fully connected visual encoder for environments with very small image inputs. (#5351)
• Colab notebooks illustrating the use of the Python API were added to the repository. (#5399)

Bug Fixes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)

• RigidBodySensorComponent now displays a warning if it's used in a way that won't generate useful observations. (#5387)
• Updated the documentation with a note saying that GridSensor does not work in 2D environments. (#5396)
• Fixed an error where sensors would not reset properly before collecting the last observation at the end of an episode. (#5375)

ml-agents / ml-agents-envs / gym-unity (Python)

• The calculation of the target entropy of SAC with continuous actions was incorrect and has been fixed. (#5372)
• Fixed an issue where the histogram stats would not be reported correctly in TensorBoard. (#5410)
• Fixed error when importing models which use the ResNet encoder. (#5358)

ML-Agents Release 17

ML-Agents Release 17

Package Versions

NOTE: It is strongly recommended that you use packages from the same release together for the best experience.

Package	Version
com.unity.ml-agents (C#)	v2.0.0
com.unity.ml-agents.extensions (C#)	v0.4.0-preview
ml-agents (Python)	v0.26.0
ml-agents-envs (Python)	v0.26.0
gym-unity (Python)	v0.26.0
Communicator (C#/Python)	v1.5.0

Breaking Changes

Minimum Version Support

• The minimum supported Unity version was updated to 2019.4. (#5166)

C# API Changes

• Several breaking interface changes were made. See the Migration Guide for more details.
• Some methods previously marked as Obsolete have been removed. If you were using these methods, you need to replace them with their supported counterpart. (#5024)
• The interface for disabling discrete actions in IDiscreteActionMask has changed. WriteMask(int branch, IEnumerable<int> actionIndices) was replaced with SetActionEnabled(int branch, int actionIndex, bool isEnabled). (#5060)
• IActuator now implements IHeuristicProvider. (#5110)
• ISensor.GetObservationShape() has been removed, and GetObservationSpec() has been added. The ITypedSensor and IDimensionPropertiesSensor interfaces have been removed. (#5127)
• ISensor.GetCompressionType() has been removed, and GetCompressionSpec() has been added. The ISparseChannelSensor interface has been removed. (#5164)
• The abstract method SensorComponent.GetObservationShape() was no longer being called, so it has been removed. (#5172)
• SensorComponent.CreateSensor() has been replaced with SensorComponent.CreateSensors(), which returns an ISensor[]. (#5181)
• The default InferenceDevice is now InferenceDevice.Default, which is equivalent to InferenceDevice.Burst. If you depend on the previous behavior, you can explicitly set the Agent's InferenceDevice to InferenceDevice.CPU. (#5175)

Model Format Changes

• Models trained with 1.x versions of ML-Agents no longer work at inference if they were trained using recurrent neural networks (#5254)
• The .onnx models input names have changed. All input placeholders now use the prefix obs_ removing the distinction between visual and vector observations. In addition, the inputs and outputs of LSTM have changed. Models created with this version are not usable with previous versions of the package (#5080, #5236)
• The .onnx models discrete action output now contains the discrete actions values and not the logits. Models created with this version are not usable with previous versions of the package (#5080)

Features Moved from com.unity.ml-agents.extensions to com.unity.ml-agents

Match3

• The Match-3 integration utilities have been moved from com.unity.ml-agents.extensions to com.unity.ml-agents. (#5259)
• Match3Sensor has been refactored to produce cell and special type observations separately, and Match3SensorComponent now produces two Match3Sensors (unless there are no special types). Previously trained models have different observation sizes and need to be retrained. (#5181)
• The AbstractBoard class for integration with Match-3 games has been changed to make it easier to support boards with different sizes using the same model. For a summary of the interface changes, please see the Migration Guide. (##5189)

Grid Sensor

• GridSensor has been refactored and moved to the main package, with changes to both sensor interfaces and behaviors. Existing GridSensor created by the extension package do not work in newer versions. Previously trained models need to be retrained. Please see the Migration Guide for more details. (#5256)

Minor Changes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)

• Updated the Barracuda package to version 1.4.0-preview(#5236)
• Added ML-Agents package settings. Now you can configure project-level ML-Agents settings in Editor > Project Settings > ML-Agents. (#5027)
• Made com.unity.modules.unityanalytics an optional dependency. (#5109)
• Made com.unity.modules.physics and com.unity.modules.physics2d optional dependencies. (#5112)
• Added support for Goal Signal as a type of observation. Trainers can now use HyperNetworks to process Goal Signal. Trainers with HyperNetworks are more effective at solving multiple tasks. (#5142, #5159, #5149)
• Modified the GridWorld environment to use the new Goal Signal feature. (#5193)
• DecisionRequester.ShouldRequestDecision() and ShouldRequestAction()methods have been added. These are used to determine whether Agent.RequestDecision() and Agent.RequestAction() are called (respectively). (#5223)
• RaycastPerceptionSensor now caches its raycast results; they can be accessed via RayPerceptionSensor.RayPerceptionOutput. (#5222)
• ActionBuffers are now reset to zero before being passed to Agent.Heuristic() and IHeuristicProvider.Heuristic(). (#5227)
• Agent now calls IDisposable.Dispose() on all ISensors that implement the IDisposable interface. (#5233)
• CameraSensor, RenderTextureSensor, and Match3Sensor now reuse their Texture2Ds, reducing the amount of memory that needs to be allocated during runtime. (#5233)
• Optimized ObservationWriter.WriteTexture() so that it doesn't call Texture2D.GetPixels32() for RGB24 textures. This results in much less memory being allocated during inference with CameraSensor and RenderTextureSensor. (#5233)

ml-agents / ml-agents-envs / gym-unity (Python)

• Some console outputs have been moved from info to debug and are no longer printed by default. If you want all messages to be printed, you can run mlagents-learn with the --debug option or add the line debug: true at the top of the yaml config file. (#5211)
• The embedding size of attention layers used when a BufferSensor is in the scene has been changed. It is now fixed to 128 units. It might be impossible to resume training from a checkpoint of a previous version. (#5272)

Bug Fixes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)

• Fixed a potential bug where sensors and actuators could get sorted inconsistently on different systems to different Culture settings. Unfortunately, this may require retraining models if it changes the resulting order of the sensors or actuators on your system. (#5194)
• Removed additional memory allocations that were occurring due to assert messages and iterating of DemonstrationRecorders. (#5246)
• Fixed a bug where agents were trying to access uninitialized fields when creating a new RayPerceptionSensorComponent on an agent. (#5261)
• Fixed a bug where the DemonstrationRecorder would throw a null reference exception if Num Steps To Record > 0 and Record was turned off. (#5274)

ml-agents / ml-agents-envs / gym-unity (Python)

• Fixed a bug where --results-dir has no effect. (#5269)
• Fixed a bug where old .pt checkpoints were not deleted during training. (#5271)

ML-Agents Release 16

ML-Agents Release 16

Package Versions

NOTE: It is strongly recommended that you use packages from the same release together for the best experience.

Package	Version
com.unity.ml-agents (C#)	v1.9.1
com.unity.ml-agents.extensions (C#)	v0.3.1-preview
ml-agents (Python)	v0.25.1
ml-agents-envs (Python)	v0.25.1
gym-unity (Python)	v0.25.1
Communicator (C#/Python)	v1.5.0

Major Changes

ml-agents / ml-agents-envs / gym-unity (Python)

• The --resume flag now supports resuming experiments with additional reward providers or loading partial models if the network architecture has changed. See here for more details. (#5213)

Bug Fixes

com.unity.ml-agents (C#)

• Fixed erroneous warnings when using the Demonstration Recorder. (#5216)

ml-agents / ml-agents-envs / gym-unity (Python)

• Fixed an issue which was causing increased variance when using LSTMs. Also fixed an issue with LSTM when used with POCA and sequence_length < time_horizon. (#5206)
• Fixed a bug where the SAC replay buffer would not be saved out at the end of a run, even if save_replay_buffer was enabled. (#5205)
• ELO now correctly resumes when loading from a checkpoint. (#5202)
• In the Python API, fixed validate_action to expect the right dimensions when set_action_single_agent is called. (#5208)
• In the GymToUnityWrapper, raise an appropriate warning if step() is called after an environment is done. (#5204)
• Fixed an issue where using one of the gym wrappers would override user-set log levels. (#5201)

ML-Agents Release 15

ML-Agents Release 15

Package Versions

NOTE: It is strongly recommended that you use packages from the same release together for the best experience.

Package	Version
com.unity.ml-agents (C#)	v1.9.0
com.unity.ml-agents.extensions (C#)	v0.3.0-preview
ml-agents (Python)	v0.25.0
ml-agents-envs (Python)	v0.25.0
gym-unity (Python)	v0.25.0
Communicator (C#/Python)	v1.5.0

Major Changes

com.unity.ml-agents (C#)

• The BufferSensor and BufferSensorComponent have been added (documentation). They allow the Agent to observe variable number of entities. For an example, see the Sorter environment. (#4909)
• The SimpleMultiAgentGroup class and IMultiAgentGroup interface have been added (documentation). These allow Agents to be given rewards and end episodes in groups. For examples, see the Cooperative Push Block, Dungeon Escape and Soccer environments. (#4923)

ml-agents / ml-agents-envs / gym-unity (Python)

• The MA-POCA trainer has been added. This is a new trainer that enables Agents to learn how to work together in groups. Configure poca as the trainer in the configuration YAML after instantiating a SimpleMultiAgentGroup to use this feature. (#5005)

Minor Changes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)

• Updated com.unity.barracuda to 1.3.2-preview. (#5084)
• Added 3D Ball to the com.unity.ml-agents samples. (#5077)

ml-agents / ml-agents-envs / gym-unity (Python)

• The encoding_size setting for RewardSignals has been deprecated. Please use network_settings instead. (#4982)
• Sensor names are now passed through to ObservationSpec.name. (#5036)

Bug Fixes

ml-agents / ml-agents-envs / gym-unity (Python)

• An issue that caused GAIL to fail for environments where agents can terminate episodes by self-sacrifice has been fixed. (#4971)
• Made the error message when observations of different shapes are sent to the trainer clearer. (#5030)
• An issue that prevented curriculums from incrementing with self-play has been fixed. (#5098)

ML-Agents Release 14

ML-Agents Release 14

Package Versions

NOTE: It is strongly recommended that you use packages from the same release together for the best experience.

Package	Version
com.unity.ml-agents (C#)	v1.8.1
com.unity.ml-agents.extensions (C#)	v0.2.0-preview
ml-agents (Python)	v0.24.1
ml-agents-envs (Python)	v0.24.01
gym-unity (Python)	v0.24.1
Communicator (C#/Python)	v1.4.0

Minor Changes

ml-agents / ml-agents-envs / gym-unity (Python)

• The cattrs version dependency was updated to allow >=1.1.0 on Python 3.8 or higher. (#4821)

Bug Fixes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)

• Fix an issue where queuing InputEvents overwrote data from previous events in the same frame.