Gensim: Integrate Conceptnet in Gensim

[souravsingh]

I have found a package for creating a semantic network of common sense knowledge- https://github.com/commonsense/conceptnet5

The word embeddings for the project are available here- https://github.com/commonsense/conceptnet-numberbatch

For a demo on Conceptnet, See here- http://conceptnet.io/

Integrating Conceptnet in Gensim would be really nice feature to have.

[markroxor]

@menshikh-iv would this be a good one to take over?

[menshikh-iv]

@markroxor yes, why not :+1:

[menshikh-iv]

So, I went through concepnet and discuss with @piskvorky, this project is really self-sufficient and integration makes no sense, for this reason, I close this issue.

[piskvorky]

Actually, I think integration would be nice, in the sense that we could support operations on ConceptNet models loaded into Gensim (what are these operations? what's the context, use-cases for this proposed feature?).

We certainly don't have to re-implement training or do anything fancy. Just use a sane interface, for using this (presumably practical) tool.

[menshikh-iv]

@piskvorky I think we probably can convert models to word2vec format and use it through KeyedVectors class. We'll add it to gensim-data soon.

[piskvorky]

So the only difference is in how the models are trained? The functionality and applicability of a trained model is identical to word2vec (like it was with gloVe)?

[menshikh-iv]

@piskvorky probably yes (maybe with small addition) we'll discuss it later with @markroxor and I'll let you know

[adrianog]

Hi

Is this still being considered?
Thanks

[menshikh-iv]

@adrianog this was already added as the ready2use pre-trained model, see https://github.com/RaRe-Technologies/gensim-data/releases/tag/conceptnet-numberbatch-17-06-300

[adrianog]

Just _superb_.
"The functionality and applicability of a trained model is identical to word2vec (like it was with gloVe)?"-> the answer to this was a "yes"? Any caveats?

[menshikh-iv]

@adrianog only thing about term construction (see "/c/en/beer" from an example from my link), see also original repo https://github.com/commonsense/conceptnet-numberbatch for more information about it. Remember that you can always extract all terms from KeyedVectors if needed.

[rlleshi]

Could you also include an English only version (_model = api.load("conceptnet-numberbatch-en-17-06-300")_) into the loading module?

[nickresearch]

Could you also include an English only version (_model = api.load("conceptnet-numberbatch-en-17-06-300")_) into the loading module?

I would like to second this suggestion--that would be fantastic. Thanks.