Smart_open: WARNING this function is deprecated, use smart_open.open instead

[mehmetilker]

Every operation I do with gensim I am seeing "2019-05-23 09:42:54,149 WARNING this function is deprecated, use smart_open.open instead"

for example:
~
model = LdaModel.load(os.path.join(TEMP_FOLDER, 'lda.model'))
model.print_topics(10)
~

I guess it started after upgrading gensim.

smart-open 1.8.3
gensim 3.7.3

[piskvorky]

That's not good, thanks for reporting. @mpenkov can you confirm?

IIRC we decided from smart_open import smart_open must remain valid and functional, because much code relies on it. Including our own, and not only in Gensim.

At the very least, deprecations should be warnings.warn, so they appear only once (not with "every operation I do", via logging). I must have missed this in a review :(

[mpenkov]

Yes, I agree that warnings.warn would have been better than logging. We'll fix this in a bugfix release ASAP.

[mehmetilker]

While waiting a fix I wanted to add ignore case for warning as it is printing that warning every item in an result array, is there any way other then "smart_open.warnings.filterwarnings('ignore')" ?
I am getting "AttributeError: 'function' object has no attribute 'warnings'" exception when I use filter...

[mpenkov]

The problem is smart_open isn't using warnings, it's using the logging module.

So if you want to suppress those logging statements, you need to do so via the functionality offered by the logging module of the standard library.

[mehmetilker]

Yet it worked with "logging.getLogger('smart_open').setLevel(logging.ERROR)"
Thanks.

[gigatesseract]

I am using the latest version of gensim (3.7.3), and I still get the same warning:
/usr/local/lib/python3.6/site-packages/smart_open/smart_open_lib.py:398: UserWarning: This function is deprecated, use smart_open.open instead. See the migration notes for details: https://github.com/RaRe-Technologies/smart_open/blob/master/README.rst#migrating-to-the-new-open-function
I have tried the following steps:

• Imported smart_open.open in my code to see if it overwrites gensim's default open, but it doesn't.
• Added the logging suppress code as mentioned by @mehmetilker , to see if it goes away, but it doesn't.

I need to optimize load times, even a single second is a boon 😅
I have a presentation due so even if Im not able to fix the issue, I should at least be able to suppress the warning.

I'd be very happy if its fixed instead of suppressing the warning. Is it possible ?

[gigatesseract]

I am using the latest version of gensim (3.7.3), and I still get the same warning:
/usr/local/lib/python3.6/site-packages/smart_open/smart_open_lib.py:398: UserWarning: This function is deprecated, use smart_open.open instead. See the migration notes for details: https://github.com/RaRe-Technologies/smart_open/blob/master/README.rst#migrating-to-the-new-open-function
I have tried the following steps:

• Imported smart_open.open in my code to see if it overwrites gensim's default open, but it doesn't.
• Added the logging suppress code as mentioned by @mehmetilker , to see if it goes away, but it doesn't.

I need to optimize load times, even a single second is a boon 😅
I have a presentation due so even if Im not able to fix the issue, I should at least be able to suppress the warning.

I'd be very happy if its fixed instead of suppressing the warning. Is it possible ?

TEMPORARY FIX:
For those of you whose still facing the problem, edit the gensim.utils .py file and import open instead of smart_open, as described below:

• Locate the gensim module, by performing the following steps:

  • Go to python shell (or python, depending on what you have, and press enter) $ python3
  • Inside the shell, import gensim. >>> import gensim
  • >>> print(gensim.__file__)
  • For me, it was '/usr/local/lib64/python3.6/site-packages/gensim/__init__.py

• Go to the gensim folder, where the __init__.py file is located (..../gensim/)

• Edit the utils.py file. In line 45 of the file (Or around there, there will be a line called from smart_open import smart_open) . Replace the line by the following line:
  from smart_open import open as smart_open
• You are good to go 😃

This will make gensim use open() instead of smart_open() everywhere.
This is wrt to the latest version of gensim(3.7.3).

If the warnings must still be supressed, you can go ahead and comment out the warnings.warn() message inside the smart_open_lib.py. (very very naive, might supresss warnings when using other modules).

@piskvorky Why have you chosen to retain smart_open instead of open? Any specific advantage?

[piskvorky]

I think @mpenkov simply didn't get to updating Gensim to use smart_open.open instead of smart_open.smart_open yet.

[mpenkov]

@gigatesseract: the warnings module of the standard library has functions for disabling warnings. Please see their documentation.

In the long term, yes, switching to the newer function is the way to go.