Gluon-nlp: New optimizer gluonnlp.optimizer.lamb.LAMB is overriding existing optimizer mxnet.optimizer.optimizer.LAMB

[leezu]

[2019-11-25T02:55:07.913Z] /var/lib/jenkins/workspace/gluon-nlp-cpu-py3/conda/cpu/py3-master/lib/python3.5/site-packages/mxnet/optimizer/optimizer.py:166: UserWarning: WARNING: New optimizer gluonnlp.optimizer.lamb.LAMB is overriding existing optimizer mxnet.optimizer.optimizer.LAMB

[leezu]

https://github.com/apache/incubator-mxnet/pull/16715

[access2rohit]

@leezu can you suggest a name for the mxnet lamb optimizer ? I will create a PR to rename it

[leezu]

@access2rohit both GluonNLP's and MXNet's optimizer.lamb are based on https://arxiv.org/pdf/1904.00962.pdf
So we may just drop the GluonNLP version.
Do you have any specific concerns about the naming of the MXNet version?

[access2rohit]

@leezu I don't have concerns about name of MXNet Lamb optimizer. I thought MXNet name was clashing with that of GluonNLP one so I might need to change in MXNet.

[leezu]

@access2rohit would it make sense to backport your MXNet PR for MXNet 1.6 release? GluonNLP will depend on MXNet 1.6, so we can't drop the current implementation.
CC @eric-haibin-lin

[leezu]

I think the changes have been backported to 1.6 on Dec 10, 2019 https://github.com/apache/incubator-mxnet/commits/v1.6.x.

[leezu]

@zburning you can consider working on this (which should fix your issue https://github.com/dmlc/gluon-nlp/issues/1085?)