Vision: Add examples of transformation to docs

[sotte]

Problem

torchvision has quite a few transforms, but I often find myself using the wrong one because the I misread/just skimmed the doc strings.

Solution

I'm suggesting to add examples / visualizations for each transform. This way it's easier to know what a given transform does.

After this improvement there should be an example (maybe a few where it makes sense) for each transform in the docs:
https://pytorch.org/docs/stable/torchvision/transforms.html#torchvision-transforms

Also, a picture is worth a thousand words ;)

What do you think?

[fmassa]

I think this is a great idea!

[sotte]

Just a few comments:
We could use the"matplotlib.sphinxext.plot_directive" extension from here.
Then for each transform something like this would do the trick and render a nice little plot:

.. plot::
   :include-source:

    import matplotlib.pyplot as plt
    import torch
    from torch.transforms import CenterCrop

    transform = CenterCrop(32)
    x = torch.rand(3, 64, 64)
    x = transform(x)
    plt.imshow(x.permute(1, 2, 0))

[fmassa]

This seems pretty clean and extensible! And also, updating the function behavior would be very simple.

I'd recommend fixing the random seed for each plot, to avoid regenerating a new image every time the docs are generated.

[sotte]

Great, I'm currently experimenting with something similar. I'll finish that and then create a PR .

I'm not sure if setting the seed prevents regeneration of the image. I'll check.

[fmassa]

It will probably regenerate the images, but they might be almost identical, so the diff size would be fairly small?

[sotte]

I'll run a bunch of experiments and get back to you.

[PyExtreme]

@sotte , Can I please pick this up? I am interested to contribute to PyTorch and this could be a good starting point.

Thanks

[sotte]

Proposal:
In order to reduce boilerplate code in the examples for the transforms add two helper functions:

• Add example_image() helper function to torchvision.utils that loads the Grace Hopper image which is already used in tests.
• Add plot_images() helper function to torchvision.utils similar to make_grid().

With the helpers in place it's quite easy to add interesting examples for each transform.

Open Questions:
Should we document TF, the transform classes or both? I have the impression that not many people know about TF. If we only add examples to TF we don't have to set the seed.
If we only add examples to TF xor transform classes, then each Transform class should link/ref the corresponding TF and the other way around so that it's easier to see the examples.
So should we document TF, transform classes, or both?

Demo:
This is what the code would look like:

from torchvision.transforms import ColorJitter
from torchvision.utils import plot_images, sample_image

transform = ColorJitter(brightness=0.5)
plot_images(*[transform(sample_image()) for _ in range(5)])

transform = ColorJitter(contrast=0.5)
plot_images(*[transform(sample_image()) for _ in range(5)])

transform = ColorJitter(saturation=0.5)
plot_images(*[transform(sample_image()) for _ in range(5)])

transform = ColorJitter(hue=0.1)
plot_images(*[transform(sample_image()) for _ in range(5)])

Just to give you an idea how the docs would look like (still with duplicate code):

Misc notes:

• Add matplotlib to docs/requirements.txt.
• Check seed for random transforms.
• I added Example in front of the code for visualizing the transform. It's not on the same level as the Example form the doc strings. I think this is fine.

[sotte]

@PyExtreme sorry, I just saw your comment. I'm already in the process of implementing this :( I'm sure we can find you another ticket to get you started contributing to pytorch.

[PyExtreme]

@sotte , please do let me know so that I can get started with it.

Thanks

[fmassa]

@sotte that's a great proposal!

Here are my thoughts:

example_image() and plot_images() in torchvision.utils

What about naming them with an underscore in the beginning to indicate that they are private? Also, the Grace Hopper image is available under test/assets I believe, which is not installed with torchvision, so this would mean that we would need to move the image somewhere else. My thinking is that those two functions are very specific to the examples, and thus having them live in torchvision.utils as being public functions might not be best.

Images for TF or transforms, or both?

I think starting with transforms would be the simplest, because we wouldn't need to manually find parameters for the transformations (and write it there).

In another node, after having another look at the transforms page in torchvision, I think we should split the page in two, and have one page dedicated for the functional transforms, and one dedicated for the module interface.

[sotte]

I created the PR #1426.

[beni20]

fyi transformations overview or list section on the top/side menu could be handy too. Examples/Images are the best only its slightly more PR effort... Currently to view all transformation options i had to scroll through the whole page:(
I was only interested in the interface/method spec dont care so much about details params

eg

• RandomJitter
• RandomPerspective
• Compose
  ...