Gluon-nlp: Importing gluonnlp giving :'ImportError: cannot import name '_get_repo_url' '

[Aashit-Sharma]

All libraries were upgraded; running on Ubuntu
A simple import of gluonnlp giving the following error

Traceback (most recent call last):
  File "lm.py", line 25, in <module>
    import gluonnlp as nlp
  File "/home/venv/gluon/lib/python3.6/site-packages/gluonnlp/__init__.py", line 23, in <module>
    from . import data
  File "/home/venv/gluon/lib/python3.6/site-packages/gluonnlp/data/__init__.py", line 23, in <module>
    from .utils import *
  File "/home/venv/gluon/lib/python3.6/site-packages/gluonnlp/data/utils.py", line 32, in <module>
    from mxnet.gluon.utils import _get_repo_url, download, check_sha1
ImportError: cannot import name '_get_repo_url'

[szha]

Currently gluonnlp requires the pre-release version of mxnet. You can try pip install --pre --upgrade mxnet

[szha]

@Aashit-Sharma let me know if there's still issue. Thanks

[vikaskumarjha9]

Earlier I ran into this issue https://github.com/apache/incubator-mxnet/issues/11911 and then I downgraded by mxnet version to 1.1.0 . After upgrading mxnet again to the solution above you have mentioned @szha I am getting the the error mentioned @Aashit-Sharma. Following are the logs.
(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ pip install mxnet-cu80==1.1.0
Collecting mxnet-cu80==1.1.0
Downloading https://files.pythonhosted.org/packages/9c/55/bcfd26fd408a4bab27bca1ef5dc1df42954509c904699a6c371d5a4c23ab/mxnet_cu80-1.1.0-py2.py3-none-manylinux1_x86_64.whl (274.9MB)
100% |████████████████████████████████| 274.9MB 172kB/s
Collecting requests==2.18.4 (from mxnet-cu80==1.1.0)
Downloading https://files.pythonhosted.org/packages/49/df/50aa1999ab9bde74656c2919d9c0c085fd2b3775fd3eca826012bef76d8c/requests-2.18.4-py2.py3-none-any.whl (88kB)
100% |████████████████████████████████| 92kB 4.1MB/s
Collecting graphviz==0.8.1 (from mxnet-cu80==1.1.0)
Downloading https://files.pythonhosted.org/packages/82/cc/668725769a1b184322019b494bbdfbb82bbef5cecc1dd8be6f778cf1422b/graphviz-0.8.1-py2.py3-none-any.whl
Collecting numpy<=1.13.3 (from mxnet-cu80==1.1.0)
Downloading https://files.pythonhosted.org/packages/0d/41/6c224571decd61c2578baedfdb0eec6283617c6679c35b20973f4e68aeaf/numpy-1.13.3-cp35-cp35m-manylinux1_x86_64.whl (16.9MB)
100% |████████████████████████████████| 16.9MB 1.3MB/s
Collecting urllib3<1.23,>=1.21.1 (from requests==2.18.4->mxnet-cu80==1.1.0)
Downloading https://files.pythonhosted.org/packages/63/cb/6965947c13a94236f6d4b8223e21beb4d576dc72e8130bd7880f600839b8/urllib3-1.22-py2.py3-none-any.whl (132kB)
100% |████████████████████████████████| 133kB 3.2MB/s
Collecting idna<2.7,>=2.5 (from requests==2.18.4->mxnet-cu80==1.1.0)
Downloading https://files.pythonhosted.org/packages/27/cc/6dd9a3869f15c2edfab863b992838277279ce92663d334df9ecf5106f5c6/idna-2.6-py2.py3-none-any.whl (56kB)
100% |████████████████████████████████| 61kB 4.9MB/s
Requirement already satisfied: certifi>=2017.4.17 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests==2.18.4->mxnet-cu80==1.1.0) (2018.11.29)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests==2.18.4->mxnet-cu80==1.1.0) (3.0.4)
Installing collected packages: urllib3, idna, requests, graphviz, numpy, mxnet-cu80
Found existing installation: urllib3 1.24.1
Uninstalling urllib3-1.24.1:
Successfully uninstalled urllib3-1.24.1
Found existing installation: idna 2.7
Uninstalling idna-2.7:
Successfully uninstalled idna-2.7
Found existing installation: requests 2.20.1
Uninstalling requests-2.20.1:
Successfully uninstalled requests-2.20.1
Found existing installation: graphviz 0.8.4
Uninstalling graphviz-0.8.4:
Successfully uninstalled graphviz-0.8.4
Found existing installation: numpy 1.14.6
Uninstalling numpy-1.14.6:
Successfully uninstalled numpy-1.14.6
Successfully installed graphviz-0.8.1 idna-2.6 mxnet-cu80-1.1.0 numpy-1.13.3 requests-2.18.4 urllib3-1.22
(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ python train_transformer.py --dataset "/home/user1/nmt_tensorflow_git/nmt/scripts/wmt16_de_en" --src_lang en --tgt_lang de --batch_size 2700 --optimizer adam --num_accumulated 16 --lr 2.0 --warmup_steps 4000 --save_dir "/home/user1/transformer_nmt/ready_to_run_transformer/saved_dir" --epochs 30 --gpus 0,1,2,3,4,5,6,7 --scaled --average_start 5 --num_buckets 20 --bucket_scheme exp --bleu 13a --log_interval 10
Traceback (most recent call last):
File "train_transformer.py", line 46, in
import gluonnlp as nlp
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/__init__.py", line 24, in
from . import data
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/data/__init__.py", line 23, in
from . import (batchify, candidate_sampler, conll, corpora, dataloader,
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/data/batchify/__init__.py", line 23, in
from . import batchify, language_model
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/data/batchify/language_model.py", line 32, in
from ..utils import slice_sequence, _slice_pad_length
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/data/utils.py", line 36, in
from mxnet.gluon.utils import _get_repo_url, download, check_sha1
ImportError: cannot import name '_get_repo_url'
(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ python train_transformer.py --dataset WMT2014BPE --src_lang en --tgt_lang de --batch_size 2700 --optimizer adam --num_accumulated 16 --lr 2.0 --warmup_steps 4000 --save_dir transformer_en_de_u512 --epochs 30 --gpus 0,1,2,3,4,5,6,7 --scaled --average_start 5 --num_buckets 20 --bucket_scheme exp --bleu 13a --log_interval 10
Traceback (most recent call last):
File "train_transformer.py", line 46, in
import gluonnlp as nlp
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/__init__.py", line 24, in
from . import data
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/data/__init__.py", line 23, in
from . import (batchify, candidate_sampler, conll, corpora, dataloader,
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/data/batchify/__init__.py", line 23, in
from . import batchify, language_model
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/data/batchify/language_model.py", line 32, in
from ..utils import slice_sequence, _slice_pad_length
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/data/utils.py", line 36, in
from mxnet.gluon.utils import _get_repo_url, download, check_sha1
ImportError: cannot import name '_get_repo_url'
(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ pip list
Package Version

---

certifi 2018.11.29
chardet 3.0.4
gluonnlp 0.5.0.post0
graphviz 0.8.1
idna 2.6
mxnet-cu80 1.1.0
numpy 1.13.3
pip 18.1
requests 2.18.4
setuptools 40.6.2
urllib3 1.22
wheel 0.32.3
(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ pip install --pre --upgrade mxnet
Collecting mxnet
Downloading https://files.pythonhosted.org/packages/3b/9d/9ff59b9a1bbd122873e5942e526543d837af4bc3ee3c1a2bba616979aff4/mxnet-1.5.0b20181217-py2.py3-none-manylinux1_x86_64.whl (27.5MB)
100% |████████████████████████████████| 27.5MB 591kB/s
Collecting requests>=2.20.0 (from mxnet)
Using cached https://files.pythonhosted.org/packages/7d/e3/20f3d364d6c8e5d2353c72a67778eb189176f08e873c9900e10c0287b84b/requests-2.21.0-py2.py3-none-any.whl
Requirement already satisfied, skipping upgrade: graphviz<0.9.0,>=0.8.1 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from mxnet) (0.8.1)
Requirement already satisfied, skipping upgrade: numpy<1.15.0,>=1.8.2 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from mxnet) (1.13.3)
Requirement already satisfied, skipping upgrade: urllib3<1.25,>=1.21.1 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests>=2.20.0->mxnet) (1.22)
Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests>=2.20.0->mxnet) (2.6)
Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests>=2.20.0->mxnet) (3.0.4)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests>=2.20.0->mxnet) (2018.11.29)
mxnet-cu80 1.1.0 has requirement requests==2.18.4, but you'll have requests 2.21.0 which is incompatible.
Installing collected packages: requests, mxnet
Found existing installation: requests 2.18.4
Uninstalling requests-2.18.4:
Successfully uninstalled requests-2.18.4
Successfully installed mxnet-1.5.0b20181217 requests-2.21.0
(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ python train_transformer.py --dataset WMT2014BPE --src_lang en --tgt_lang de --batch_size 2700 --optimizer adam --num_accumulated 16 --lr 2.0 --warmup_steps 4000 --save_dir transformer_en_de_u512 --epochs 30 --gpus 0,1,2,3,4,5,6,7 --scaled --average_start 5 --num_buckets 20 --bucket_scheme exp --bleu 13a --log_interval 10
Illegal instruction (core dumped)
(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ python train_transformer.py --dataset "/home/user1/nmt_tensorflow_git/nmt/scripts/wmt16_de_en" --src_lang en --tgt_lang de --batch_size 2700 --optimizer adam --num_accumulated 16 --lr 2.0 --warmup_steps 4000 --save_dir "/home/user1/transformer_nmt/ready_to_run_transformer/saved_dir" --epochs 30 --gpus 0,1,2,3,4,5,6,7 --scaled --average_start 5 --num_buckets 20 --bucket_scheme exp --bleu 13a --log_interval 10
Illegal instruction (core dumped)

[vikaskumarjha9]

I am sorry the formatting here, I am new to posting logs. I will attach it as a file.
log.txt

[szha]

@vikaskumarjha9 Sorry to hear that it's still happening to you. From the log, it looks like you have both mxnet and mxnet-cu80, and the mxnet-cu80 is of older version. That might be the cause of this issue.

[vikaskumarjha9]

I have removed mxnet library, now I have only mxnet-cu80 version1.1.0 as my CUDA version is 8+. I am getting this error.

(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ python train_transformer.py --dataset WMT2014BPE --src_lang en --tgt_lang de --batch_size 2700 --optimizer adam --num_accumulated 16 --lr 2.0 --warmup_steps 4000 --save_dir transformer_en_de_u512 --epochs 30 --gpus 0,1,2,3,4,5,6,7 --scaled --average_start 5 --num_buckets 20 --bucket_scheme exp --bleu 13a --log_interval 10
Traceback (most recent call last):
File "train_transformer.py", line 45, in
from mxnet import gluon
ImportError: cannot import name 'gluon'

I tried installing gluon package but no help. Also I have gluonnlp-0.5.0.post0 installed, does this work with mxnet-cu80? Pls help.

[szha]

@vikaskumarjha9 GluonNLP 0.5.0 depends on MXNet 1.3.0+, so in this case you can try pip install --upgrade mxnet-cu80.

[vikaskumarjha9]

Thank you for prompt reply, I updated it to 1.3.1 but now I am getting this error again.
(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ pip install --upgrade mxnet-cu80
Collecting mxnet-cu80
Using cached https://files.pythonhosted.org/packages/b0/a4/f4ab7885416a5b752782492dba335f58d942a917a7259826689f552d9af6/mxnet_cu80-1.3.1-py2.py3-none-manylinux1_x86_64.whl
Requirement already satisfied, skipping upgrade: numpy<1.15.0,>=1.8.2 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from mxnet-cu80) (1.13.3)
Requirement already satisfied, skipping upgrade: graphviz<0.9.0,>=0.8.1 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from mxnet-cu80) (0.8.1)
Requirement already satisfied, skipping upgrade: requests>=2.20.0 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from mxnet-cu80) (2.21.0)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests>=2.20.0->mxnet-cu80) (2018.11.29)
Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests>=2.20.0->mxnet-cu80) (3.0.4)
Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests>=2.20.0->mxnet-cu80) (2.6)
Requirement already satisfied, skipping upgrade: urllib3<1.25,>=1.21.1 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests>=2.20.0->mxnet-cu80) (1.22)
Installing collected packages: mxnet-cu80
Found existing installation: mxnet-cu80 1.1.0
Uninstalling mxnet-cu80-1.1.0:
Successfully uninstalled mxnet-cu80-1.1.0
Successfully installed mxnet-cu80-1.3.1
(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ python train_transformer.py --dataset WMT2014BPE --src_lang en --tgt_lang de --batch_size 2700 --optimizer adam --num_accumulated 16 --lr 2.0 --warmup_steps 4000 --save_dir transformer_en_de_u512 --epochs 30 --gpus 0,1,2,3,4,5,6,7 --scaled --average_start 5 --num_buckets 20 --bucket_scheme exp --bleu 13a --log_interval 10
Illegal instruction (core dumped)

Am I doing something wrong ?

[vikaskumarjha9]

Oops sorry, here you go.
log2.txt

[szha]

@vikaskumarjha9 I'm guessing this is related to some instruction set support of the runtime environment. Would you mind trying to update this to MXNet's latest nightly build and see if the issue persists? You can achieve it by running pip install --pre --upgrade mxnet-cu80.

If the issue persists with the 1.5.0b nightly versions, we will need to take a closer look. In that case, could you download and run this script, so that we can examine the supported instructions set on your machine?

[gulelishibhat]

tyhsxvwqwasavsrfrhenwr gfweq5ywhnsqgwefsqWHWGES6XYEWS @HCEKDFNCCRYESDYSH3WBQAWF2Q @B DHEVSVFYSB #1538