Fairseq: AdaFactor to save GPU memory?

Created on 20 Sep 2018  路  3Comments  路  Source: pytorch/fairseq

Tensor2Tensor has AdaFactor to drastically reduce the GPU memory usage. I believe it would be helpful for FairSeq to have this by default.

enhancement
Source
picture AranKomat
👍1

Most helpful comment

Working on this

picture luciodery on 4 Dec 2018
👍6

All 3 comments

Good idea!

myleott picture myleott on 20 Sep 2018

Working on this

luciodery picture luciodery on 4 Dec 2018
👍6

See Adafactor here :
https://github.com/pytorch/fairseq/blob/master/fairseq/optim/adafactor.py

luciodery picture luciodery on 25 Jan 2019
❤3
Was this page helpful?
0 / 5 - 0 ratings

Related issues

ajesujoba picture ajesujoba  路  3Comments
ashim95 picture ashim95  路  3Comments
mjpost picture mjpost  路  3Comments
Raghava14 picture Raghava14  路  3Comments
neel04 picture neel04  路  3Comments