Opennmt-py: how to open the demo.train.pt file?

[XiangZhu0718]

after the first step:python preprocess.py, generating three files: demo.train.pt, demo.valid.pt and demo.vocab.pt. how can I look up these files and know what are the vocabularies? thk you.

[Henry-E]

import torch
vocab = torch.load("./demo.vocab.pt")
print(vocab)
[('src', <torchtext.vocab.Vocab at 0x2b95b1735b38>),
 ('tgt', <torchtext.vocab.Vocab at 0x2b95b1735b38>)]

Now you have two torchtext vocab objects that you can get the source and target vocabs from in a dict by checking freqs, itos or stoi

vocab[0][1].freqs
Counter({'__start_name__': 1,
         'The': 1,
         'Vaults': 1,
         ...})