Tensor2tensor: embedding layer

[anglil]

In the symbol modality, it doesn't seem to have an embedding layer. Instead, symbols are mapped to hidden dimension directly from a one-hot vector. I'm wondering what's an easy way to plug in a pretrained embedding layer that has a different dimension from the hidden layer?

[rsepassi]

You could subclass SymbolModality and override _body_input_depth to be something other than self._model_hparams.hidden_size.

[leshow123]

A follow-up question: can't the REAL modality be used to plug-in a word embedding (since they are vectors of real numbers)?