Setuptools: imp is deprecated

[ghost]

Originally reported by: jaraco (Bitbucket: jaraco, GitHub: jaraco)

---

According to the docs, imp is deprecated since Python 3.4. Yet pkg_resources relies on it. In particular, it relies on the acquire_lock and release_lock functions, which don't have an equivalent in importlib.

The docs do say, "Changed in version 3.3: The locking scheme has changed to per-module locks for the most part. A global import lock is kept for some critical tasks, such as initializing the per-module locks."

What does that mean for the usage in pkg_resources? Is it possible to eliminate the usage of locks on Python 3.3+ in declare_namespace and fixup_namespace_packages?

This issue was revealed in Pull Request 129, but the resolution was to keep it and the deprecation warnings. What's the proper solution here?

---

• Bitbucket: https://bitbucket.org/pypa/setuptools/issue/479

[mgedmin]

pkg_resources isn't the only place that imports imp

pkg_resources/__init__.py:    import imp as _imp
setuptools/command/build_ext.py:import imp
setuptools/command/install_lib.py:import imp
setuptools/depends.py:import imp
setuptools/depends.py:from imp import PKG_DIRECTORY, PY_COMPILED, PY_SOURCE, PY_FROZEN
setuptools/site-patch.py:                import imp  # Avoid import loop in Python >= 3.3

Python 3.6 upgrades the PendingDeprecationWarning to a full DeprecationWarning, which in practical terms means nothing, only some psychological pressure not to forget about this issue. ;)

[5j9]

Is it acceptable to temporarily suppress the warnings in setuptools using warnings.catch_warnings or does it need to be a thread-safe or some other solution?

[jaraco]

I'm not sure what you're asking. Are you asking if you can suppress the warnings in your code? Well, sure. If you're asking about suppressing the warnings in setuptools, it feels like that might just kick the can down the road. And besides, most applications have these warnings disabled by default anyway, so if a user is enabling them, I would expect them to want to see them.

[5j9]

I enable these warnings in my own code to fix issues before they become breaking changes. But when the number of them grows, it becomes hard to keep an eye on them and detecting the new ones become harder. That's why I thought about suppressing some of those that are already reported and known to be rather harmless for the time being...

I'm not sure if this whole thing is a good idea or not, and if it is, whether it is better to be done through the upstream libraries (setuptools in this case) or downstream.

Anyway, I can see why setuptools owners might be reluctant to suppress this warning. I'll find another way.

Thanks for the reply.

[giampaolo]

Right now I use this hack in psutil in order to avoid this (and others) deprecation warnings issued by setuptools, which is annoying:
https://github.com/giampaolo/psutil/blob/master/setup.py#L17-L24

with warnings.catch_warnings():
    warnings.simplefilter("ignore")
    try:
        import setuptools
        from setuptools import setup, Extension
    except ImportError:
        setuptools = None
        from distutils.core import setup, Extension

[hugovk]

Setuptools now only supports non-EOL Python versions: 2.7 and 3.4+.

• imp is deprecated since version 3.4, in favour of importlib.
  https://docs.python.org/3/library/imp.html

• importlib is available in Python 2.7 as "a minor subset of what is available in the more full-featured package of the same name from Python 3.1 that provides a complete implementation of import. What is here has been provided to help ease in transitioning from 2.7 to 3.1."
  https://docs.python.org/2.7/library/importlib.html

Can imp now be replaced in Setuptools?

• https://github.com/pypa/setuptools/search?q=imp

[jaraco]

@hugovk: You're welcome to give it a try. Perhaps the project should try simply removing the locks and hope that nothing breaks. Please feel free to investigate.

[asottile]

I started work on the easier parts of this in #1798 -- would appreciate a review :)