Gluon-nlp: [word2vec] Add word2vec binary load function

[imgarylai]

I'm willing to add this feature to gluon-nlp.

Our lab is using gensim to load word2vec binary file and we want to get rid of it. @szha and I have discussed with issue before. This would be great that gluon-nlp can have an interface to load our own pre-trained fastText binary.

@szha @leezu, please let me know how can I help you guys for this.

[szha]

Hi @imgarylai. Thanks for offering to help! For fastText format, there's a load_fasttext_format method in the FasttextEmbeddingModel class that can be used without gensim. Feel free to give it a try and see if it fulfills your need.

[szha]

Alternatively, the word2vec format is still missing. Would that be something you're interested in too?

[imgarylai]

Yes. I have changed the title. 😝

[leezu]

Thanks @imgarylai

I guess the best way forward would be to just specialize / overwrite the from_file for the Word2Vec class or to introduce a new method there https://github.com/dmlc/gluon-nlp/blob/master/src/gluonnlp/embedding/token_embedding.py#L961

(For fastText this was instead added in the model.train.embedding API as the binary format does not contain the "final" vectors but rather the vectors for all words / ngrams that must still be combined. But this does not apply in the W2V case.)

[imgarylai]

Hello @leezu,

I probably have several questions needed help. I have been using gluon-nlp for a while but never checked every details in this project. I will appreciate all your helps before I get used to everything!

The first question I have is about the api design. Classes implemented in the src/gluonnlp/embedding/token_embedding.py require a source to be initialized. In this case, are we going to support w2v in source list only for this from_file method?

Or would it be expected do the similar idea like load_fasttext_format in FasttextEmbeddingModel? In this case, people can load their own w2v (probably either text or binary).

[szha]

@imgarylai Word2Vec class was written in the way that requires source so that automatic downloading is easier. You can think of the from_file/load_w2v_binary method as another constructor independent of the source abstraction.

[imgarylai]

Move to #620