Incubator-mxnet: Optimizer wish list

Created on 22 Dec 2017  路  6Comments  路  Source: apache/incubator-mxnet

Optimizer plays a fundamental role in machine learning. This issue tracks the optimizers that people have requested for support in MXNet. We can also comment below to recommend new optimizers. It's inspired by the thread in https://discuss.gluon.ai/t/topic/3714.

@szha @szhengac

Call for Contribution Optimizer
Source
picture sxjscience
👍15

Most helpful comment

AdamW, SGDW

picture kobenaxie on 23 Dec 2017
👍10

All 6 comments

NeumannOptimizer

VikingMew picture VikingMew on 22 Dec 2017

AdamW, SGDW

kobenaxie picture kobenaxie on 23 Dec 2017
👍10

working on FTML

ZiyueHuang picture ZiyueHuang on 30 Dec 2017
👍3
  • [ ] Limited-memory BFGS (L-BFGS)
jasonyu1996 picture jasonyu1996 on 14 Aug 2018
👍3
sxjscience picture sxjscience on 28 Jun 2019
👍2

AdaBound
The authors call it as fast as Adam and as good as SGD.
So it might a good one.

braindotai picture braindotai on 16 Jul 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

phunterlau picture phunterlau  路  3Comments
zy-huang picture zy-huang  路  3Comments
seongkyun picture seongkyun  路  3Comments
realbns2008 picture realbns2008  路  3Comments
Ajoo picture Ajoo  路  3Comments