Pytorch-lightning: Suggestion to add the default interval of scheduler in the documentation

[DKandrew]

📚 Documentation

The default interval of scheduler is per epoch. However, it is not explicitly mentioned in the documentation. I have to dig into the code and figure it out.
https://github.com/PyTorchLightning/pytorch-lightning/blob/0914873bc2f588ae592c896460c0c9e6577324cf/pytorch_lightning/trainer/optimizers.py#L86

I understand it is sort of common sense for people who are already familiar with training the network in PyTorch, but if lightning is aiming for the more general public, then adding this explicitly into the documentation may help people to better understand the training pipeline of their network.

Maybe we need to add all the default ("interval", "frequency", "reduce_on_plateau", "monitor") into the documentation explicitly.

By the way, I am not sure if the default of scheduler in TF is in step or epoch, but some popular networks (e.g. DeepLabV3) using step instead of epoch as the default interval for their scheduler,

[Borda]

@DKandrew Mind sending a PR with this docs update?
@awaelchli any other thoughts?

[awaelchli]

Agree, it's a good idea to document the defaults :)

[DKandrew]

I also find another issue about the documentation for this function.

In the html file, function configure_optimizers() returns

Dictionary, with an ‘optimizer’ key and (optionally) a ‘lr_scheduler’ key.

The value of the 'lr_scheduler' key is actually a SINGLE dictionary that contains.

{
    'scheduler': lr_scheduler,
    'interval': 'step'  # or 'epoch'
    'monitor': 'val_f1',
    'reduce_on_plateau': False, 
    'frequency': x
}

I highlighted the SINGLE because this is what it really does
https://github.com/PyTorchLightning/pytorch-lightning/blob/0914873bc2f588ae592c896460c0c9e6577324cf/pytorch_lightning/trainer/optimizers.py#L41
This is not clearly mentioned in the doc.

I think this dictionary structure also applies to the Two lists case where the second is a list of LR schedulers or the lr_dict,
and applies to another return type

Tuple of dictionaries as described, with an optional ‘frequency’ key.

, right?

[DKandrew]

I create a local branch and make a first draft of the documentation, but I cannot upload this branch to the github, it reports 403, no permission. How should I do it?

[Borda]

I create a local branch and make a first draft of the documentation, but I cannot upload this branch to the github, it reports 403, no permission. How should I do it?

you need to fork the repo and create the branch there, then you can create a PR from your forked repo to the main repo... it just how GH works... :]

[DKandrew]

you need to fork the repo and create the branch there, then you can create a PR from your forked repo to the main repo... it just how GH works... :]

Hi Borda, Thank you for the instructions! I make the PR.

By the way, do you know how to break a line while enumerating things in the doc? I get warning that my line is too long at line 1012 and 1013, but if I break the line, the make html will create warning and the list does not display properly. I checked the Sphinx and reST but did not find any information.