Xla: regarding summary writer + google colab + TPU

[vainaijr]

hello,

the SummaryWriter does not work when I do this in google colab

!pip install
http://storage.googleapis.com/pytorch-tpu-releases/tf-1.13/torch-1.0.0a0+1d94a2b-cp36-cp36m-linux_x86_64.whl
http://storage.googleapis.com/pytorch-tpu-releases/tf-1.13/torch_xla-0.1+5622d42-cp36-cp36m-linux_x86_64.whl

and when I upgrade torch,

import torch_xla gives error

is it possible to make both of them work, so I can use TPU on google colab, and use
SummaryWriter from torch.utils.Tensorboard, also is it possible to have one GAN example with TPU

[dlibenzi]

The torch_xla and torch wheels are bound together as torch_xla interfaces torch via a _C++_ API.
Unfortunately the VMs in Colab are running an older version of the runtime in the TPU VM.

[vainaijr]

I am able to use torch_xla in Google Colab but how to transfer generator and discriminator to TPU

[dlibenzi]

Keep in mind that the wheels that today work with _Colab_, are very old and they support the old _PyTorch/XLA_ implementation which was based on _PyTorch_ JIT.
The new _PyTorch/XLA_ uses the new _XTen_ (lazy tensor) approach which is much more flexible than the old approach, as it allows to run essentially every _PyTorch_ model, but redirection currently TPU-unsupported _PyTorch_ operations, to _PyTorch_ CPU.
Unfortunately in order to use the new _PyTorch/XLA_ implementation, you will need that the _Colab_ VMs will be based on newer TPU service code.
We are working on that!

[vainaijr]

is there any updated wheel, that works with torch 1.1, as the wheel that I mentioned above uninstalls torch 1.1?

this is the message I get

Successfully uninstalled torch-1.1.0
Successfully installed torch-1.0.0a0+1d94a2b torch-xla-0.1+5622d42

[dlibenzi]

Unfortunately no at this time.
Sorry about that.

[adrianog]

Hi

Thanks for an amazing library.
"Unfortunately in order to use the new PyTorch/XLA implementation, you will need that the Colab VMs will be based on newer TPU service code."

What version of the TPU service code is required?
I notice that the VM are now running 1.14.0-rc1 - does that change things at all?

[jysohn23]

The Colab's TPUs software version is not yet up to date, but we should tentatively be updating the Cloud TPU software versions that Colab uses in the next coming weeks so that you should be able to use Colab + TPU + PyTorch/XLA in the future.

[adrianog]

Do you know how one can check the Colab's TPUs software version?
I could not find anything obvious on tf.contrib.tpu

[jysohn23]

Sorry about the late reply!

Do you know how one can check the Colab's TPUs software version?
I could not find anything obvious on tf.contrib.tpu

The TPU's software version should be the exact same one as the one printed from

import tensorflow as tf; tf.__version__

This may be confusing for PyTorch/XLA's users as we're PyTorch not TF, but we do depend on XRT which is built into TF. For colab's purpose we are planning on cutting a release of a pair of torch and torch_xla wheels that are compatible with the TF 1.15 TPU Software Version.