Opennmt-py: -emb_file could match -emb_file_enc, -emb_file_dec

[pinzhenliao]

Hi,
Thank you very much for the great tool.

When I followed the instruction on page http://opennmt.net/OpenNMT-py/FAQ.html#how-do-i-use-pretrained-embeddings-e-g-glove to use Pretrained embeddings (e.g. GloVe),

when I run

./tools/embeddings_to_torch.py -emb_file "glove_dir/glove.6B.300d.txt" -dict_file "data/demo.vocab.pt" -output_file "data/embeddings"

I got this error:

embeddings_to_torch.py: error: ambiguous option: -emb_file could match -emb_file_enc, -emb_file_dec

You might want to update the FAQ page.

[vivekmishra1991]

Were you able to solve this? Any workaround?

[pltrdy]

The doc is actually out of date. One must explicitely specify which vector file you want to use, for both the encoder and the decoder.

If you want to use the same vectors, just set:
./tools/embeddings_to_torch.py -emb_file_enc "glove_dir/glove.6B.300d.txt" -emb_file_dec "glove_dir/glove.6B.300d.txt" -dict_file "data/demo.vocab.pt" -output_file "data/embeddings"

[pltrdy]

@wingsyuan indeed.

[wingsyuan]

@pltrdy Hi,Thanks very much for your help!
Here is my another questions:

• enc: 20925 match, 8793 missing, (70.41%)
  
  
  
  • dec: 20923 match, 13342 missing, (61.06%)
  
  

Filtered embeddings:
* enc: torch.Size([29718, 300])
* dec: torch.Size([34265, 200])

how does the system handle the missing embedding? Are the misssing parts tokens embedding under the random initialization at the beginning of the trainning? then both the miss and unmiss are update when trainning?

what does the Filtered embeddings mean? they are of different dim.. Can I use them directly?

[pltrdy]

I just checked, missing tokens are in the vocabulary and the embeddings. They get 0 valued vector, which is maybe not a super good idea, random would have been better I guess.

By default, embedding weights will be updated during training, you can set -fix_word_vecs_enc / -fix_word_vecs_dec to keep (all) vectors from updating.

The embeddings dimensions are [vocabulary_size x vec_dim], the vec_dim depends on your embedding file. You probably gave a 200 dim src emb and 300 dim tgt emb.

---

Where default value is set: https://github.com/OpenNMT/OpenNMT-py/blob/master/tools/embeddings_to_torch.py#L70

[pltrdy]

Yup, it should be ok.

The word order isn't the same tho, i.e. one word is associated to a token (i.e. integer value) based on the vocabulary (basically index in a list of words). A given word may, or may not, have different tokens in src vocab and tgt vocab.

[wingsyuan]

Thanks very much!@pltrdy

[pltrdy]

Ur welcome :)