Pytorch-lightning: Lightning/Transformers tutorial contribution?

Created on 27 Oct 2019 · 7Comments · Source: PyTorchLightning/pytorch-lightning

First off, thank you for your hard work on this package! It's proven to be _very_ helpful for fully leveraging multi-GPU training of large-scale language models.

Is your feature request related to a problem? Please describe.
I'm using the (also incredible) transformers package by the folks at HuggingFace to do a number of tasks, including text classification. I realized that training these models can be quite computationally expensive, even on decently-sized single-GPU instances. I found out about this repo and was (with relative ease) able to train the same model over 4 GPUs and 16-bit precision much, _much_ more easily than training with vanilla PyTorch.

I was wondering if there was any interest in a contributed tutorial that specifically goes through the process of integrating pytorch-lightning with transformers, which seems to have quite a bit of staying power and success within the realm of applied NLP. Regardless, thanks again!

Describe the solution you'd like
N/A?

Describe alternatives you've considered
N/A?

Additional context
N/A?

enhancement help wanted

Source

dataframing

👍5 ❤1

Most helpful comment

I would suggest creating a tutorial folder with notebooks. Later we can automatically generate documentation from these notebooks, see example from readthedocs

Borda on 30 Oct 2019

👍2

All 7 comments

yes! that would be amazing. would welcome a PR!

williamFalcon on 27 Oct 2019

Neat! I noticed that the current domain template (for GANs here) is a Python script — is that preferred over a full notebook?

dataframing on 27 Oct 2019

@williamFalcon bumping this — I've got a more in-depth notebook that shows how to use transformers with pytorch-lightning, but I can make it a Python script, instead. Let me know!

dataframing on 30 Oct 2019

I would suggest creating a tutorial folder with notebooks. Later we can automatically generate documentation from these notebooks, see example from readthedocs

Borda on 30 Oct 2019

👍2

@dataframing go ahead and make a notebook! That would be awesome

williamFalcon on 3 Nov 2019

👍1

Sounds good! Apologies for the delay here; had a busy week. Should be able to get something out soon 👍

dataframing on 4 Nov 2019

@dataframing any updates? in the meantime, we can use the tutorial i put together on medium and the COLAB in the readme

williamFalcon on 21 Jan 2020

Was this page helpful?

0 / 5 - 0 ratings