Staged-recipes: Thought on testing pure-python packages

[msarahan]

I have been dealing with updating an internal recipe that is nominally pure python, but has many dependencies. The number of dependencies has changed over time, and not all of these dependencies are imported by simply importing the main module. Stuff is pretty broken with those dependencies not listed in the conda recipe.

This is avoidable by running the test suite - but maybe just flake8 would be enough to catch these? Anyway, small thought about why we should not just do simple import tests...

[ocefpaf]

@msarahan Linux packagers have a policy to always run the full test suit no matter what. I would love to do that but we are limited by our resources. (Although I am pretty sure we could get away with the majority of the packages here.)

Not sure if if I understand your flake8 idea. How would that be used to catch unspecified dependencies in the recipe?

[msarahan]

flake8 catches unresolved imports. I think if you were to run flake8 on the installed package folder, you might get errors for the missing libraries.

[ocefpaf]

flake8 catches unresolved imports. I think if you were to run flake8 on the installed package folder, you might get errors for the missing libraries.

Got it. Worth a try!

[msarahan]

To clarify, this is a feature of pyflakes. flake8 unifies pyflakes and pep8. We could just use pyflakes, since flake8 might get tripped up on pep8 errors.

[jakirkham]

Ideally we would write a few custom tests to exercise the library, but I don't think it is a practical use of manpower at this point and can just as likely missing things. I like your flake8 idea @msarahan. Just want to make sure we aren't trying to really lint people's codes. Don't want it complaining about things like long line length or unused imports and needing patches or workarounds for that.

@sigmavirus24, you have a pretty strong interest and involvement in various code quality tools. Do you have any recommendations for us?

[sigmavirus24]

So flake8 alone won't warn you that you're importing something that's not importable. For that, you need to install flake8's extension: hacking (maintained by OpenStack's developers... I also happen to be a core reviewer on the project).

pyflakes only warns on _unused_ imports, e.g., code like:

import pdb
import collections

d = collections.defaultdict(list)
d[1].append(2)

Would provide a warning about importing pdb without using it. If instead you had code like:

import requests

r = requests.get('https://httpbin.org/get')
print(r.status_code)

But didn't have requests installed, pyflakes wouldn't complain, but hacking would (and only as a side-effect of some odd checks that hacking runs).

---

As for finding dependencies that we haven't installed... I think we could take a bunch of steps towards figuring this out:

1. Looking for requirements.txt files. This is the simplest case (if someone has a file that lists all the dependencies) but it's really not going to work all the time.
2. Run python setup.py egg_info and introspect {package_name}.egg_info/requires.txt to see what the setup.py thinks is necessary
3. Check setup.cfg for metadata about requirements, e.g., flake8's setup.cfg
4. Build a wheel and then unpack it to determine what metadata it has around requirements.

I can help build this tooling, and we should probably inject setuptools for the latter portion of these options in a similar manner to how pip does on installation.

(Head's up, I'm also involved in packaging efforts in python)

[sigmavirus24]

This also brings up conditional requirements. Flake8, for example, only needs two of its dependencies on certain versions of python. How should those be expressed by recipes?

[jakirkham]

Any more thoughts on this or should we close?

[CJ-Wright]

Would https://github.com/ericdill/depfinder be of interest?