Release/4.0.0 (#6240)

* Release 4.0.0 (#6220)

Author: maryamziaa

com.unity.ml-agents/Documentation~/Readme.md renamed to Readme.md

@@ -2,27 +2,27 @@
 
 [![docs badge](https://img.shields.io/badge/docs-reference-blue.svg)](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest)
 
-[![license badge](https://img.shields.io/badge/license-Apache--2.0-green.svg)](https://github.com/Unity-Technologies/ml-agents/blob/release_22/LICENSE.md)
+[![license badge](https://img.shields.io/badge/license-Apache--2.0-green.svg)](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/LICENSE.md)
 
 ([latest release](https://github.com/Unity-Technologies/ml-agents/releases/tag/latest_release)) ([all releases](https://github.com/Unity-Technologies/ml-agents/releases))
 
 **The Unity Machine Learning Agents Toolkit** (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents. We provide implementations (based on PyTorch) of state-of-the-art algorithms to enable game developers and hobbyists to easily train intelligent agents for 2D, 3D and VR/AR games. Researchers can also use the provided simple-to-use Python API to train Agents using reinforcement learning, imitation learning, neuroevolution, or any other methods. These trained agents can be used for multiple purposes, including controlling NPC behavior (in a variety of settings such as multi-agent and adversarial), automated testing of game builds and evaluating different game design decisions pre-release. The ML-Agents Toolkit is mutually beneficial for both game developers and AI researchers as it provides a central platform where advances in AI can be evaluated on Unity’s rich environments and then made accessible to the wider research and game developer communities.
 
 ## Features
-- 17+ [example Unity environments](Learning-Environment-Examples.md)
+- 17+ [example Unity environments](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Learning-Environment-Examples.html)
 - Support for multiple environment configurations and training scenarios
 - Flexible Unity SDK that can be integrated into your game or custom Unity scene
 - Support for training single-agent, multi-agent cooperative, and multi-agent competitive scenarios via several Deep Reinforcement Learning algorithms (PPO, SAC, MA-POCA, self-play).
 - Support for learning from demonstrations through two Imitation Learning algorithms (BC and GAIL).
-- Quickly and easily add your own [custom training algorithm](Python-Custom-Trainer-Plugin.md) and/or components.
+- Quickly and easily add your own [custom training algorithm](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-Custom-Trainer-Plugin.html) and/or components.
 - Easily definable Curriculum Learning scenarios for complex tasks
 - Train robust agents using environment randomization
 - Flexible agent control with On Demand Decision Making
 - Train using multiple concurrent Unity environment instances
-- Utilizes the [Inference Engine](Inference-Engine.md) to provide native cross-platform support
-- Unity environment [control from Python](Python-LLAPI.md)
-- Wrap Unity learning environments as a [gym](Python-Gym-API.md) environment
-- Wrap Unity learning environments as a [PettingZoo](Python-PettingZoo-API.md) environment
+- Utilizes the [Inference Engine](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Inference-Engine.html) to provide native cross-platform support
+- Unity environment [control from Python](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-LLAPI.html)
+- Wrap Unity learning environments as a [gym](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-Gym-API.html) environment
+- Wrap Unity learning environments as a [PettingZoo](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-PettingZoo-API.html) environment
 
 ## Releases & Documentation
 
@@ -31,15 +31,15 @@
 
 The table below shows our latest release, including our `develop` branch which is under active development and may be unstable. A few helpful guidelines:
 
-- The [Versioning page](Versioning.md) overviews how we manage our GitHub releases and the versioning process for each of the ML-Agents components.
+- The [Versioning page](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Versioning.html) overviews how we manage our GitHub releases and the versioning process for each of the ML-Agents components.
 - The [Releases page](https://github.com/Unity-Technologies/ml-agents/releases) contains details of the changes between releases.
-- The [Migration page](Migrating.md) contains details on how to upgrade from earlier releases of the ML-Agents Toolkit.
+- The [Migration page](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Migrating.html) contains details on how to upgrade from earlier releases of the ML-Agents Toolkit.
 - The `com.unity.ml-agents` package is [verified](https://docs.unity3d.com/2020.1/Documentation/Manual/pack-safe.html) for Unity 2020.1 and later. Verified packages releases are numbered 1.0.x.
 
-| **Version** | **Release Date** | **Source** | **Documentation** | **Download** | **Python Package** | **Unity Package** |
-|:-----------:|:---------------:|:----------:|:-----------------:|:------------:|:------------------:|:-----------------:|
-| **Release 22** | **October 5, 2024** | **[source](https://github.com/Unity-Technologies/ml-agents/tree/release_22)** | **[docs](https://unity-technologies.github.io/ml-agents/)** | **[download](https://github.com/Unity-Technologies/ml-agents/archive/release_22.zip)** | **[1.1.0](https://pypi.org/project/mlagents/1.1.0/)** | **[3.0.0](https://docs.unity3d.com/Packages/[email protected]/manual/index.html)** |
-| **develop (unstable)** | -- | [source](https://github.com/Unity-Technologies/ml-agents/tree/develop) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/develop/com.unity.ml-agents/Documentation~/index.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/develop.zip) | -- | -- |
+|      **Version**       |  **Release Date**   |                                  **Source**                                   |                                                 **Documentation**                                                  |                                      **Download**                                      |                  **Python Package**                   |                                   **Unity Package**                                   |
+|:----------------------:|:-------------------:|:-----------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------:|:-----------------------------------------------------:|:-------------------------------------------------------------------------------------:|
+|     **Release 23**     | **August 15, 2025** | **[source](https://github.com/Unity-Technologies/ml-agents/tree/release_23)** |              **[docs](https://docs.unity3d.com/Packages/com.unity.ml-agents@4.0/manual/index.html)**               | **[download](https://github.com/Unity-Technologies/ml-agents/archive/release_23.zip)** | **[1.1.0](https://pypi.org/project/mlagents/1.1.0/)** |                                       **4.0.0**                                       |
+| **develop (unstable)** |         --          |    [source](https://github.com/Unity-Technologies/ml-agents/tree/develop)     | [docs](https://github.com/Unity-Technologies/ml-agents/tree/develop/com.unity.ml-agents/Documentation~/index.md)   |    [download](https://github.com/Unity-Technologies/ml-agents/archive/develop.zip)     |                         --                            |                                          --                                           |
 
 
 
@@ -77,12 +77,12 @@ Additionally, if you use the MA-POCA trainer in your research, we ask that you c
 * [Introduction to ML-Agents by Huggingface](https://huggingface.co/learn/deep-rl-course/en/unit5/introduction)
 * [Community created ML-Agents projects](https://discussions.unity.com/t/post-your-ml-agents-project/816756)
 * [ML-Agents models on Huggingface](https://huggingface.co/models?library=ml-agents)
-* [Blog posts](Blog-posts.md)
+* [Blog posts](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Blog-posts.html)
 * [Discord](https://discord.com/channels/489222168727519232/1202574086115557446)
 
 ## Community and Feedback
 
-The ML-Agents Toolkit is an open-source project and we encourage and welcome contributions. If you wish to contribute, be sure to review our [contribution guidelines](CONTRIBUTING.md) and [code of conduct](https://github.com/Unity-Technologies/ml-agents/blob/release_22/CODE_OF_CONDUCT.md).
+The ML-Agents Toolkit is an open-source project and we encourage and welcome contributions. If you wish to contribute, be sure to review our [contribution guidelines](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/CONTRIBUTING.html) and [code of conduct](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/CODE_OF_CONDUCT.md).
 
 For problems with the installation and setup of the ML-Agents Toolkit, or discussions about how to best setup or train your agents, please create a new thread on the [Unity ML-Agents discussion forum](https://discussions.unity.com/tag/ml-agents). Be sure to include as many details as possible to help others assist you effectively. If you run into any other problems using the ML-Agents Toolkit or have a specific feature request, please [submit a GitHub issue](https://github.com/Unity-Technologies/ml-agents/issues).
 

colab/Colab_UnityEnvironment_1_Run.ipynb

@@ -32,7 +32,7 @@
    },
    "source": [
     "# ML-Agents Open a UnityEnvironment\n",
-    "<img src=\"https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/images/image-banner.png?raw=true\" align=\"middle\" width=\"435\"/>"
+    "<img src=\"https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/com.unity.ml-agents/Documentation~/images/image-banner.png?raw=true\" align=\"middle\" width=\"435\"/>"
    ]
   },
   {

colab/Colab_UnityEnvironment_2_Train.ipynb

@@ -22,7 +22,7 @@
    },
    "source": [
     "# ML-Agents Q-Learning with GridWorld\n",
-    "<img src=\"https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/images/gridworld.png?raw=true\" align=\"middle\" width=\"435\"/>"
+    "<img src=\"https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/com.unity.ml-agents/Documentation~/images/gridworld.png?raw=true\" align=\"middle\" width=\"435\"/>"
    ]
   },
   {
@@ -190,7 +190,7 @@
     "id": "pZhVRfdoyPmv"
    },
    "source": [
-    "The [GridWorld](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Learning-Environment-Examples.md#gridworld) Environment is a simple Unity visual environment. The Agent is a blue square in a 3x3 grid that is trying to reach a green __`+`__ while avoiding a red __`x`__.\n",
+    "The [GridWorld](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Learning-Environment-Examples.html#gridworld) Environment is a simple Unity visual environment. The Agent is a blue square in a 3x3 grid that is trying to reach a green __`+`__ while avoiding a red __`x`__.\n",
     "\n",
     "The observation is an image obtained by a camera on top of the grid.\n",
     "\n",

colab/Colab_UnityEnvironment_3_SideChannel.ipynb

@@ -23,7 +23,7 @@
    },
    "source": [
     "# ML-Agents Use SideChannels\n",
-    "<img src=\"https://raw.githubusercontent.com/Unity-Technologies/ml-agents/release_22_docs/docs/images/3dball_big.png\" align=\"middle\" width=\"435\"/>"
+    "<img src=\"https://raw.githubusercontent.com/Unity-Technologies/ml-agents/release/4.0.0/docs/images/3dball_big.png\" align=\"middle\" width=\"435\"/>"
    ]
   },
   {
@@ -176,7 +176,7 @@
     "## Side Channel\n",
     "\n",
     "SideChannels are objects that can be passed to the constructor of a UnityEnvironment or the `make()` method of a registry entry to send non Reinforcement Learning related data.\n",
-    "More information available [here](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Python-API.md#communicating-additional-information-with-the-environment)\n",
+    "More information available [here](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-LLAPI.html#communicating-additional-information-with-the-environment)\n",
     "\n",
     "\n",
     "\n"
@@ -189,7 +189,7 @@
    },
    "source": [
     "### Engine Configuration SideChannel\n",
-    "The [Engine Configuration Side Channel](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Python-API.md#engineconfigurationchannel) is used to configure how the Unity Engine should run.\n",
+    "The [Engine Configuration Side Channel](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-LLAPI.html#engineconfigurationchannel) is used to configure how the Unity Engine should run.\n",
     "We will use the GridWorld environment to demonstrate how to use the EngineConfigurationChannel."
    ]
   },
@@ -282,7 +282,7 @@
    },
    "source": [
     "### Environment Parameters Channel\n",
-    "The [Environment Parameters Side Channel](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Python-API.md#environmentparameters) is used to modify environment parameters during the simulation.\n",
+    "The [Environment Parameters Side Channel](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-LLAPI.html#environmentparameters) is used to modify environment parameters during the simulation.\n",
     "We will use the GridWorld environment to demonstrate how to use the EngineConfigurationChannel."
    ]
   },
@@ -419,7 +419,7 @@
    },
    "source": [
     "### Creating your own Side Channels\n",
-    "You can send various kinds of data between a Unity Environment and Python but you will need to [create your own implementation of a Side Channel](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Custom-SideChannels.md#custom-side-channels) for advanced use cases.\n"
+    "You can send various kinds of data between a Unity Environment and Python but you will need to [create your own implementation of a Side Channel](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Custom-SideChannels.html#custom-side-channels) for advanced use cases.\n"
    ]
   },
   {

colab/Colab_UnityEnvironment_4_SB3VectorEnv.ipynb

@@ -7,7 +7,7 @@
    },
    "source": [
     "# ML-Agents run with Stable Baselines 3\n",
-    "<img src=\"https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/images/image-banner.png?raw=true\" align=\"middle\" width=\"435\"/>"
+    "<img src=\"https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/com.unity.ml-agents/Documentation~/images/image-banner.png?raw=true\" align=\"middle\" width=\"435\"/>"
    ]
   },
   {

com.unity.ml-agents/CHANGELOG.md

@@ -6,7 +6,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project adheres to
 [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [4.0.0] - 2025-08-15
 ### Major Changes
 #### com.unity.ml-agents (C#)
 - Upgraded to Inference Engine 2.2.1 (#6212)

com.unity.ml-agents/Documentation~/Inference-Engine.md

@@ -6,7 +6,7 @@ The ML-Agents Toolkit allows you to use pre-trained neural network models inside
 
 Inference Engine supports [all Unity runtime platforms](https://docs.unity3d.com/Manual/PlatformSpecific.html).
 
-Scripting Backends : Inference Engine is generally faster with **IL2CPP** than with **Mono** for Standalone builds. In the Editor, It is not possible to use Inference Engine with GPU device selected when Editor Graphics Emulation is set to **OpenGL(ES) 3.0 or 2.0 emulation**. Also there might be non-fatal build time errors when target platform includes Graphics API that does not support **Unity Compute Shaders**.
+Scripting Backends : Inference Engine is generally faster with **IL2CPP** than with **Mono** for Standalone builds. In the Editor, It is not possible to use Inference Engine with GPU device selected when Editor Graphics Emulation is set to **OpenGL(ES) 3.0 or 2.0 emulation**. Also, there might be non-fatal build time errors when target platform includes Graphics API that does not support **Unity Compute Shaders**.
 
 In cases when it is not possible to use compute shaders on the target platform, inference can be performed using **CPU** or **GPUPixel** Inference Engine backends.
 
@@ -18,7 +18,7 @@ When using a model, drag the model file into the **Model** field in the Inspecto
 
 # Unsupported use cases
 ## Externally trained models
-The ML-Agents Toolkit only supports the models created with our trainers. Model loading expects certain conventions for constants and tensor names. While it is possible to construct a model that follows these conventions, we don't provide any additional help for this. More details can be found in [TensorNames.cs](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/com.unity.ml-agents/Runtime/Inference/TensorNames.cs) and [SentisModelParamLoader.cs](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/com.unity.ml-agents/Runtime/Inference/SentisModelParamLoader.cs).
+The ML-Agents Toolkit only supports the models created with our trainers. Model loading expects certain conventions for constants and tensor names. While it is possible to construct a model that follows these conventions, we don't provide any additional help for this. More details can be found in [TensorNames.cs](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/com.unity.ml-agents/Runtime/Inference/TensorNames.cs) and [SentisModelParamLoader.cs](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/com.unity.ml-agents/Runtime/Inference/SentisModelParamLoader.cs).
 
 If you wish to run inference on an externally trained model, you should use Inference Engine directly, instead of trying to run it through ML-Agents.