v0.4.5: Patch release

Use correct input size for splits (#1284)

* Use correct input size for splits

* Use smarter partitioning

v0.4.4: Patch release

[Doc] round_robin_gradients (#1261)

* Fix docstring

* Make screenshots clickable for easier viewing

* Navigation menu in alphabetical order; More clicable screenshots

* Rename 1Cycle doc

* Tweak naming

* Remove no longer used flag

* ZeRO3 Offload release

* Single GPU results

* Rearrange figures

* Single GPU text

* tweak intro

* zero3-offload section

* Add asynchronous i/o docs

* Fix print_per_steps doc

* Document round_robin_gradients

* Tweak description

* Trigger CI

v0.4.3: Patch release

revert part of #1220 (#1221)

https://github.com/microsoft/DeepSpeed/pull/1220 fixed the leak, but lead to another problem. reverting that part so that we could do release and will work on it after the release.

@jeffra

v0.4.2: Patch release

clean up logging (#1190)

Co-authored-by: Jeff Rasley <[email protected]>

v0.4.1: Patch release

remove torchvision dependency (#1178)

DeepSpeed v0.4.0

DeepSpeed v0.4.0

• [Press release] DeepSpeed: Accelerating large-scale model inference and training via system optimizations and compression
• New inference API inference setup
• DeepSpeed Inference: Multi-GPU inference with customized inference kerenls and quantization support
  • Getting Started with DeepSpeed for Inferencing Transformer based Models
• Mixture-of-Quantization: A novel quantization approach for reducing model size with minimal accuracy impact
  • MoQ tutorial for more details.

DeepSpeed v0.3.16

v0.3.16 Release notes

• Full precision (fp32) support for ZeRO Stage2 and Stage3 (#1004)
• 1-bit LAMB optimizer (#970)
• Refactor param_dict to config (#1008)
• [ZeRO Infinity] Allow Init to take a dict for the deepspeed config (#983)
• Bug fix for zero-1 w.r.t. reduce-scatter (#907)
• Use amp autocast in ZeRO3 linear (#990)
• Add find_unused_parameters option to DeepSpeedEngine (#945)
• Relax dataset type check in deepspeed io (#1012)

Special thanks to our contributors: @stas00, @SeanNaren, @sdtblck, @wbuchwalter, @ghosthamlet, @zhujiangang,

DeepSpeed v0.3.15

v0.3.15 Release notes

• ZeRO-Infinity release allowing nvme offload and more!

• Deprecated cpu_offload in config JSON, see JSON docs for more details.

• Automatic external parameter registration, more details in the ZeRO 3 docs.

• Several bug fixes for ZeRO stage 3

DeepSpeed v0.3.14

Notes to come

DeepSpeed v0.3.13

v0.3.13 Release notes

Combined release notes since Jan 12th v0.3.10 release

• ZeRO 3 Offload (#834)
• more detailed notes to come