Parlai: Getting error when running examples/display_data.py

[tibnb545]

Bug description
After running

git clone https://github.com/facebookresearch/ParlAI.git
cd ParlAI; python setup.py develop

to install ParlAI, when trying to run the first example script

python examples/display_data.py -t babi:task1k:1

there is an error which is shown below.

Logs

Traceback (most recent call last):
  File "examples/display_data.py", line 23, in <module>
    display_data(opt)
  File "/home/user1/ParlAI/parlai/scripts/display_data.py", line 46, in display_data
    world.parley()
  File "/home/user1/ParlAI/parlai/core/worlds.py", line 342, in parley
    agents[0].observe(validate(acts[1]))
  File "/home/user1/ParlAI/parlai/core/teachers.py", line 444, in observe
    self.metrics.evaluate_response(observation, self.lastY)
  File "/home/user1/ParlAI/parlai/core/metrics.py", line 714, in evaluate_response
    self.add(f'bleu-{k}', BleuMetric.compute(prediction, labels, k))
  File "/home/user1/ParlAI/parlai/core/metrics.py", line 585, in add
    self._data[key] = self._data.get(key) + value
TypeError: unsupported operand type(s) for +: 'NoneType' and 'NoneType'

[stephenroller]

Thanks for reporting.

I suspect the issue is that nltk isn’t installed, but the system should work around that.

I’ll put out a fix shortly

[tgmerritt]

Having similar issue -

Traceback (most recent call last):
  File "/Users/tyler/.pyenv/versions/3.7.2/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/Users/tyler/ParlAI/parlai/core/worlds.py", line 1322, in run
    world.parley()
  File "/Users/tyler/ParlAI/parlai/core/worlds.py", line 349, in parley
    agents[0].observe(validate(acts[1]))
  File "/Users/tyler/ParlAI/parlai/core/teachers.py", line 445, in observe
    self.metrics.evaluate_response(observation, self.lastY)
  File "/Users/tyler/ParlAI/parlai/core/metrics.py", line 722, in evaluate_response
    self.add(f'bleu-{k}', BleuMetric.compute(prediction, labels, k))
  File "/Users/tyler/ParlAI/parlai/core/metrics.py", line 591, in add
    self._buffer[key] = self._buffer.get(key) + value
TypeError: unsupported operand type(s) for +: 'NoneType' and 'NoneType'

Running python examples/train_model.py -t babi:task10k:1 -mf /tmp/babi_memnn -bs 1 -nt 4 -eps 5 -m memnn --no-cuda from the Quick Start guide

The training hasn't errored out just yet ~

I seem to have nltk already installed - but missing 'wheel'

pip3 install nltk
Requirement already satisfied: nltk in ./.pyenv/versions/3.7.2/lib/python3.7/site-packages/nltk-3.4.5-py3.7.egg (3.4.5)
Requirement already satisfied: six in ./.pyenv/versions/3.7.2/lib/python3.7/site-packages (from nltk) (1.14.0)
Could not build wheels for nltk, since package 'wheel' is not installed.
Could not build wheels for six, since package 'wheel' is not installed.

I installed the wheel package and I'm still seeing the same error.

I added some logging to check values passed into this function:

[loading fbdialog data:/Users/tyler/ParlAI/data/bAbI/tasks_1-20_v1-2/en-valid-10k-nosf/qa1_train.txt]
Before failing line - self is {} and key is exs and value is 1
Before failing line - self is {'exs': SumMetric(1)} and key is accuracy and value is 1
Before failing line - self is {'exs': SumMetric(1), 'accuracy': ExactMatchMetric(1)} and key is f1 and value is 1
Before failing line - self is {'exs': SumMetric(1), 'accuracy': ExactMatchMetric(1), 'f1': F1Metric(1)} and key is bleu-4 and value is None

[Meai1]

EDIT: I simply added a line checking whether the value is None at metrics.py:590 and now some samples seem to run fine:

https://github.com/facebookresearch/ParlAI/blob/f7f78ad068d8a719f404ddc909ca32a5a9fa4ef4/parlai/core/metrics.py#L590-L594

        if value is not None:  # <---- new line
            if self._threadsafe and self._worker:
                self._buffer[key] = self._buffer.get(key) + value
            else:
                self._data[key] = self._data.get(key) + value

[stephenroller]

What version of python were you on?

[Meai]

3.8.2

[github-actions[bot]]

This issue has not had activity in 30 days. Please feel free to reopen if you have more issues. You may apply the "never-stale" tag to prevent this from happening.

[stephenroller]

Still needs to be fixed.