File tree 2 files changed +8
-8
lines changed
2 files changed +8
-8
lines changed Original file line number Diff line number Diff line change 14
14
* Improve the stability of ` ADOPT ` optimizer. (#294 )
15
15
* [ Note] ( https://github.com/iShohei220/adopt?tab=readme-ov-file#update-on-nov-22-2024 )
16
16
* Support a new projection type ` random ` for ` GaLoreProjector ` . (#294 )
17
+ * Implement ` DeMo ` optimizer. (#300 , #301 )
18
+ * [ Decoupled Momentum Optimization] ( https://arxiv.org/abs/2411.19870 )
19
+ * Implement ` Muon ` optimizer. (#302 )
20
+ * [ MomentUm Orthogonalized by Newton-schulz] ( https://github.com/KellerJordan/Muon )
21
+ * Implement ` ScheduleFreeRAdam ` optimizer. (#304 )
22
+ * Implement ` LaProp ` optimizer. (#304 )
23
+ * [ Separating Momentum and Adaptivity in Adam] ( https://arxiv.org/abs/2002.04839 )
24
+ * Support ` Cautious ` variant to ` LaProp ` , ` AdamP ` , ` Adopt ` optimizers. (#304 ).
17
25
18
26
### Refactor
19
27
Original file line number Diff line number Diff line change 2
2
3
3
### Feature
4
4
5
- * Implement ` DeMo ` optimizer. (#300 , #301 )
6
- * [ Decoupled Momentum Optimization] ( https://arxiv.org/abs/2411.19870 )
7
- * Implement ` Muon ` optimizer. (#302 )
8
- * [ MomentUm Orthogonalized by Newton-schulz] ( https://github.com/KellerJordan/Muon )
9
- * Implement ` ScheduleFreeRAdam ` optimizer. (#304 )
10
- * Implement ` LaProp ` optimizer. (#304 )
11
- * [ Separating Momentum and Adaptivity in Adam] ( https://arxiv.org/abs/2002.04839 )
12
- * Support ` Cautious ` variant to ` LaProp ` , ` AdamP ` , ` Adopt ` optimizers. (#304 ).
You can’t perform that action at this time.
0 commit comments