Transformers: Installation example #2 fails: cannot import name 'glue_compute_metrics'

[evanweissburg]

🐛 Bug

I am having issues with the official installation procedure, where running python -m pytest -sv ./examples fails with an opaque error message (below).

To Reproduce

Steps to reproduce the behavior:

1. Create virtualenv
2. Install Pytorch (pip install torch==1.2.0+cpu torchvision==0.4.0+cpu -f https://download.pytorch.org/whl/torch_stable.html)
3. Install transformers (pip install transformers)
4. Install pytest (pip install pytest)
5. Run python -m pytest -sv ./transformers/tests/; no tests fail
6. Run python -m pytest -sv ./examples/; fails requiring tensorboardX
7. Install tensorboardX (pip install tensorboardX)
8. Run python -m pytest -sv ./examples/; fails with message:

==================================== ERRORS ====================================
__________________ ERROR collecting examples/test_examples.py __________________
ImportError while importing test module '/home/evancw/Projects/transformers/examples/test_examples.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
examples/test_examples.py:30: in <module>
    import run_glue
examples/run_glue.py:49: in <module>
    from transformers import glue_compute_metrics as compute_metrics
E   ImportError: cannot import name 'glue_compute_metrics'
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.64s ===============================

Expected behavior

I would expect the tests to pass.

Environment

• OS: Ubuntu 18.04.3 LTS
• Python version: 3.6.8
• PyTorch version: 1.2.0 + CPU
• PyTorch Transformers version (or branch): 2.0.0
• Using GPU ? No
• Distributed of parallel setup ? No
• Any other relevant information: System is completely clean before pip installs

Additional context

Please let me know if there is any more information I can provide.

[vikrant094]

@evanweissburg Hi, i got the same error..have you found any solution???

[LysandreJik]

Hello! I believe you must have sklearn installed in order to pass these tests. Please let me know if it doesn't work while having sklearn installed.

[vikrant094]

yeah got it .... i guess we need to run pip install -r ./examples/requirements.txt

[LysandreJik]

Indeed!

[paullovessearch]

Could we add this to the getting started documentation?

pip install -r ./examples/requirements.txt

[yuvalkirstain]

Had the same issue. Probably a problem with sklearn. Installed with conda and it was fixed.