Smart_open: file_smart_open missing mode 'rb' caused error

[cosmoschen94]

Line 668 in def file_smart_open(fname, mode='rb', encoding=None, errors=DEFAULT_ERRORS) in smart_open_lib.py

try:  # TODO need to fix this place (for cases with r+ and so on)
        raw_mode = {'r': 'rb', 'w': 'wb', 'a': 'ab'}[mode]
except KeyError:
        raw_mode = mode

Should include r+ cases. Otherwise, when mode == 'rb', this will cause KeyError

[mpenkov]

Thank you for reporting this, but I'm not sure what the issue is here. If mode is rb, then a KeyError _is_ raised, but it gets caught immediately below.

Reading files in binary mode also works as expected:

(smartopen)sergeyich:smart_open misha$ python
Python 3.6.3 (default, Oct  4 2017, 06:09:15)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.37)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import smart_open
>>> fin = smart_open.smart_open('setup.py', 'rb')
>>> binary = fin.read()
>>> repr(binary[:10])
"b'#!/usr/bin'"

Could you please clarify what the problem is? What error are you seeing, and what exactly are you doing to cause that error?

[cosmoschen94]

I was trying to load a pickle file from my local file system by using
model = KeyedVectors.load('input_model.p', mmap='r')
and then KeyError was caught, which prevents me from loading the model

[cosmoschen94]

after I changed
raw_mode = {'r': 'rb', 'w': 'wb', 'a': 'ab'}[mode]
to
raw_mode = {'r': 'rb', 'rb':'rb', 'w': 'wb', 'a': 'ab'}[mode]
I was able to load the model

[mpenkov]

Thank you for providing us more details.

What is KeyedVectors? It isn't part of the smart_open library, from what I know. How is your code using smart_open?

The KeyError should definitely be caught... Could you please paste your relevant code, and the full stack trace here?

[menshikh-iv]

@mpenkov KeyedVector is class from gensim that used for storing *2vec vectors (code)

Need to test this part more carefully and investigate this problem.
Thanks for report @cosmoschen94, can you share your input_model.p file with us (for reproducing your error)?

[menshikh-iv]

I tried to reproduce this (for py27 and py36 versions), but I see no bug, everything is working fine.

versions:

smart-open==1.5.5
gensim==3.2.0

import gensim.downloader as api

model = api.load("glove-twitter-25")
model.most_similar("hello")  # works fine


# save model
model.save("input_model.p")

# load without mmap
loaded_model = model.load("input_model.p")
loaded_model.most_similar("hello")  # works fine


# load with mmap
loaded_model = model.load("input_model.p", mmap="r")
loaded_model.most_similar("hello")  # works fine

For this reason, I close this issue.