Flair: "TypeError: expected str, bytes or os.PathLike object, not NoneType"

[jneto04]

Hello, I'm trying to run Flair on the Google Cloud. But every time I run a test I get this error:

Traceback (most recent call last):
File "test_flair.py", line 1, in
from flair.data import Sentence
File "/usr/local/lib/python3.6/dist-packages/flair/__init__.py", line 2, in
from . import models
File "/usr/local/lib/python3.6/dist-packages/flair/models/__init__.py", line 1, in
from .sequence_tagger_model import SequenceTagger
File "/usr/local/lib/python3.6/dist-packages/flair/models/sequence_tagger_model.py", line 12, in
import flair.embeddings
File "/usr/local/lib/python3.6/dist-packages/flair/embeddings.py", line 7, in
import gensim
File "/usr/local/lib/python3.6/dist-packages/gensim/__init__.py", line 5, in
from gensim import parsing, corpora, matutils, interfaces, models, similarities, summarization, utils # noqa:F401
File "/usr/local/lib/python3.6/dist-packages/gensim/parsing/__init__.py", line 4, in
from .preprocessing import (remove_stopwords, strip_punctuation, strip_punctuation2, # noqa:F401
File "/usr/local/lib/python3.6/dist-packages/gensim/parsing/preprocessing.py", line 40, in
from gensim import utils
File "/usr/local/lib/python3.6/dist-packages/gensim/utils.py", line 44, in
from smart_open import smart_open
File "/usr/local/lib/python3.6/dist-packages/smart_open/__init__.py", line 1, in
from .smart_open_lib import *
File "/usr/local/lib/python3.6/dist-packages/smart_open/smart_open_lib.py", line 45, in
from boto.compat import BytesIO, urlsplit, six
File "/usr/lib/python3/dist-packages/boto/__init__.py", line 1216, in
boto.plugin.load_plugins(config)
File "/usr/lib/python3/dist-packages/boto/plugin.py", line 92, in load_plugins
for file in glob.glob(os.path.join(directory, '*.py')):
File "/usr/lib/python3.6/posixpath.py", line 80, in join
a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

The test I'm running on is this:

from flair.data import Sentence
from flair.models import SequenceTagger

sentence = Sentence('I love Berlin .')

tagger = SequenceTagger.load('ner')

tagger.predict(sentence)

The settings for my VM instance are:
-> 8 vCPUs with 32gb
-> 1 NVIDIA Tesla K80
-> Ubuntu 16.04
-> Cuda 10.0

Thank you!

[alanakbik]

Hello @jneto04 thanks for reporting this. It seems like the error is already thrown at the import statement and is somehow thrown during the import of the gensim library.

To debug, could you print the version of gensim etc. that are installed in your virtualenvironment on google cloud?

[jneto04]

I'm using Python3.6 with pip3.6. With the "pip3.6 freeze" command I noticed that I was using "pytorch-pretrained-bert == 0.3.0.". So I did an update and installed the version "pytorch-pretrained-bert == 0.4.0".

Then I ran the test again, but I continue with the same error. Here's a list of the modules I'm using:

(The "->" arrow means that that module is required for FLAIR according to the "requirements.txt" file in FLAIR github.)

atomicwrites==1.2.1
attrs==18.2.0
beautifulsoup4==4.4.1
blinker==1.3
boto==2.38.0
boto3==1.9.74
botocore==1.12.74
bz2file==0.98
chardet==2.3.0
cloud-init==18.4
command-not-found==0.3
configobj==5.0.6
cryptography==1.2.3
cycler==0.10.0
decorator==4.3.0
defer==1.0.6
--> Deprecated==1.2.4
docutils==0.14
flair==0.4.0
future==0.17.1
--> gensim==3.4.0
google-compute-engine==2.8.4
html5lib==0.999
--> hyperopt==0.1.1
idna==2.0
Jinja2==2.8
jmespath==0.9.3
jsonpatch==1.10
jsonpointer==1.9
kiwisolver==1.0.1
language-selector==0.1
lxml==3.5.0
MarkupSafe==0.23
--> matplotlib==3.0.0
more-itertools==5.0.0
--> mpld3==0.3
networkx==2.2
numpy==1.15.4
oauthlib==1.0.3
pluggy==0.7.1
prettytable==0.7.2
py==1.7.0
pyasn1==0.1.9
pycups==1.9.73
pycurl==7.43.0
pygobject==3.20.0
PyJWT==1.3.0
pymongo==3.7.2
pyparsing==2.3.0
pyserial==3.0.1
--> pytest==3.6.4
python-apt==1.1.0
b1+ubuntu0.16.4.2
python-dateutil==2.7.5
python-debian==0.1.27
python-systemd==231
--> pytorch-pretrained-bert==0.4.0
pyxdg==0.25
PyYAML==3.11
regex==2018.11.22
requests==2.9.1
s3transfer==0.1.13
scikit-learn==0.20.2
scipy==1.2.0
screen-resolution-extra==0.0.0
--> segtok==1.5.7
six==1.10.0
--> sklearn==0.0
smart-open==1.7.1
--> sqlitedict==1.6.0
ssh-import-id==5.5
system-service==0.3
--> torch==1.0.0
--> tqdm==4.26.0
--> typing==3.6.4
ufw==0.35
unattended-upgrades==0.1
urllib3==1.24.1
virtualenv==16.2.0
wrapt==1.10.11
xkit==0.0.0

Thankful!

[alanakbik]

Thanks - one more question: do you have a virtual environment or did you install flair directly on the machine? If not, could you try setting up a fresh virtual environment and installing flair there?

[jneto04]

I have a virtual machine (virtual environment) in the Google Cloud.

However, I created a new virtual machine and reinstalled everything that is needed, but I continue with the error. :(

Thanks again!

[jneto04]

I was able to solve the problem. Apparently the problem was in the Gensim, triggered by the Boto. What I did was:

sudo pip3.6 uninstall boto
sudo pip3.6 install boto

And everything worked perfectly!

Thankful!

[alanakbik]

Ah great - thanks for sharing the solution!

[ghost]

what about python ?
i am running my code in python. But got a problem.

TypeError: expected str, bytes or os.PathLike object, not NoneType

[alanakbik]

Can you post a minimal code example to reproduce the error?

[ghost]

actually i am too running a project of fruit recognization on kraggle kernel where i uploaded a zip folder of the dataset containing train test and valid folder and among them different types of fruits folder but when i am running the code it is giving error as expexted str, bytes or os. PathLike objects, not list.......

i used the code...
trainpath=os.listdir('../input/data/data/train/')

[ShivamPanchal]

Anyone resolved it? @sayan231 @Ammy143

[Virako]

I was a similar error with python and Django, the problem was an empty folder. When you are reading folders, python check dirname, if the dirname is None, return error.

Steps to found empty folders are:

• Review the error trace, and search what is the library that are calling to function dirname in /usr/lib/python3.6/posixpath.py (in the example /usr/lib/python3/dist-packages/boto/plugin.py", line 92, in load_plugins)
• Inside the function (load_plugins), put debugger trace and search what is the folder that call to dirname function and send None param

[nesspll]

Turn out I forgot to add: __init__.py in a folder I was calling as a package, that solved the issue