Opennmt-py: Question: visualize attention

[iwatobipen]

Dear developer,

I am newbie of openmnt-py. The library is very helpful for me. And I have a question.
I would like to visualize attention like following tutorial.
http://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html
Are there any optional argument to get attention information when I try to run translation?
Any comments or advice is appreciated.
Best regards,

iwatobipen

[srush]

So you can use the following snippet to run a translation model from a notebook.

The attention matrix for each sentence is in translations[0].attn , you can then visualize it or print it however you would like.

import onmt
import onmt.io
import onmt.translate
import onmt.ModelConstructor
from collections import namedtuple

# Load the model.
Opt = namedtuple('Opt', ['model', 'data_type', 'reuse_copy_attn', "gpu"])

opt = Opt("/home/srush/Downloads/model_acc_51.33_ppl_12.74_e20.pt", "text",False, 0)
fields, model, model_opt =  onmt.ModelConstructor.load_test_model(opt,{"reuse_copy_attn":False})

# Test data
data = onmt.io.build_dataset(fields, "text", "/home/srush/data/sum3/valid.article.filter.txt", None, use_filter_pred=False)
data_iter = onmt.io.OrderedIterator(
        dataset=data, device=0,
        batch_size=1, train=False, sort=False,
        sort_within_batch=True, shuffle=False)
# Translator
translator = onmt.translate.Translator(model, fields,
                                           beam_size=5,
                                           n_best=1,
                                           global_scorer=None,
                                           cuda=True)

builder = onmt.translate.TranslationBuilder(
        data, translator.fields,
        1, False, None)

for j, batch in enumerate(data_iter):
        batch_data = translator.translate_batch(batch, data)
        translations = builder.from_batch(batch_data)
        print("src:", " ".join(translations[0].src_raw))
        print()
        print("tgt:", " ".join(translations[0].pred_sents[0]))

[iwatobipen]

Thank you for your quick response! I could get attn matrix.
I appreciate your kind help!

[ruohoruotsi]

I've tried this snippet, but realize that I need to have a global_scorer which you have set to None.

I get this error:

File "/home/ubuntu/github/OpenNMT-py/onmt/translate/Beam.py", line 112, in advance
self.global_scorer.update_global_state(self)
AttributeError: 'NoneType' object has no attribute 'update_global_state'

How do you update global state without a global_scorer and what object should I be using here? Cheers!

[srush]

(@sebastianGehrmann think this is from your change. Need to support global scorer None. )

@ruohoruotsi , quick fix global_scorer=GNMTGlobalScorer(0, 0, "none", "none")

[ruohoruotsi]

Great! Thank you I'll try that out ... I was messing around with this that I found in translate.py

scorer = onmt.translate.GNMTGlobalScorer(opt.alpha,
                                         opt.beta,
                                         opt.coverage_penalty,
                                         opt.length_penalty)

BTW, @srush great job with ONMT, I'm new in the last few weeks, and loving every part of using it!

[ghost]

some namespace has changed. e.g. io -> inputters
could use some update here

[esalesky]

This works for me with the current version:

~~~
import onmt
import onmt.inputters
import onmt.translate
import onmt.model_builder
from collections import namedtuple

Load the model.

Opt = namedtuple('Opt', ['model', 'data_type', 'reuse_copy_attn', "gpu"])

opt = Opt("/Users/esalesky/projects/models/lstm_clean_acc_51.81_ppl_15.58_e13.pt", "text",False, 0)
fields, model, model_opt = onmt.model_builder.load_test_model(opt,{"reuse_copy_attn":False})

Test data

data = onmt.inputters.build_dataset(fields, "text", None, use_filter_pred=False, src_path='/Users/esalesky/projects/data/fisher/test.es')
data_iter = onmt.inputters.OrderedIterator(
dataset=data, device='cuda',
batch_size=1, train=False, sort=False,
sort_within_batch=True, shuffle=False)

Translator

translator = onmt.translate.Translator(model, fields,
beam_size=5,
n_best=1,
global_scorer=onmt.translate.GNMTGlobalScorer(0, 0, "none", "none"),
gpu=True)

builder = onmt.translate.TranslationBuilder(
data, translator.fields,
1, False, None)

for j, batch in enumerate(data_iter):
batch_data = translator.translate_batch(batch, data)
translations = builder.from_batch(batch_data)
print("src:", " ".join(translations[0].src_raw))
print("tgt:", " ".join(translations[0].pred_sents[0]))
print("idx:",str(j))
print("-----")
~~~

[Huijun-Cui]

thanks, is there any document about the onmt for detail ? the code above is not available now

[ruohoruotsi]

I used the above code successfully:
https://github.com/Niger-Volta-LTI/yoruba-adr/blob/master/src/plot_att_weights.py (generate matrices)
https://github.com/Niger-Volta-LTI/yoruba-adr/blob/master/src/Plot_Attention_Weights.ipynb (load up and plot images for paper!)

[anubrata]

Hi, I am pretty new to seq2seq models and OpenNMT-py. I am using OpenNMT for a summarization problem and was able to train a basic model using the examples. However, I tried to visualize the attention weights using the code mentioned in this thread and I am getting the following error:

AttributeError: 'dict' object has no attribute 'seek'. You can only torch.load from a file that is seekable. Please pre-load the data into a buffer like io.BytesIO and try to load from it instead.

This works for me with the current version:

```
import onmt
import onmt.inputters
import onmt.translate
import onmt.model_builder
from collections import namedtuple

Load the model.

Opt = namedtuple('Opt', ['model', 'data_type', 'reuse_copy_attn', "gpu"])

opt = Opt("/Users/esalesky/projects/models/lstm_clean_acc_51.81_ppl_15.58_e13.pt", "text",False, 0)

The error is occurring particularly after executing this line below.

fields, model, model_opt = onmt.model_builder.load_test_model(opt,{"reuse_copy_attn":False})
```

Could you please help me with this issue?

[Rajashan]

The namespace seems to have changed again, does anyone have a workaround?

[gregdurrett]

For anyone who discovers this thread in 2020, the -attn_debug flag passed to onmt_translate prints a pretty nice visualization of the attention matrix without having to write any additional code.