Salt: integration.modules.test_pip.PipModuleTest

[dwoz]

integration.modules.test_pip.PipModuleTest.test_chained_requirements__absolute_file_path
--------------------------------------------------------------------------------
develop failed salt-fedora-28-py3
--------------------------------------------------------------------------------
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
string indices must be integers
................................................................................

Traceback (most recent call last):
  File "/tmp/kitchen/testing/tests/integration/modules/test_pip.py", line 275, in test_chained_requirements__absolute_file_path
    self.assertEqual(ret['retcode'], 0)
TypeError: string indices must be integers

integration.modules.test_pip.PipModuleTest.test_requirements_as_list__non_absolute_file_path
--------------------------------------------------------------------------------
develop failed salt-fedora-28-py3
--------------------------------------------------------------------------------
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
string indices must be integers
................................................................................

Traceback (most recent call last):
  File "/tmp/kitchen/testing/tests/integration/modules/test_pip.py", line 247, in test_requirements_as_list__non_absolute_file_path
    self.assertEqual(ret['retcode'], 0)
TypeError: string indices must be integers

[waynew]

Also failing here:

develop failed salt-ubuntu-1804-py3

string indices must be integers

Traceback (most recent call last):
  File "/tmp/kitchen/testing/tests/integration/modules/test_pip.py", line 182, in test_requirements_as_list_of_chains__cwd_not_set__absolute_file_path
    self.assertEqual(ret['retcode'], 0)
TypeError: string indices must be integers

[waynew]

And these cases:

• integration.modules.test_pip.PipModuleTest.test_requirements_as_list__non_absolute_file_path
• integration.modules.test_pip.PipModuleTest.test_pip_install_multiple_editables
• integration.modules.test_pip.PipModuleTest.test_requirements_as_list__absolute_file_path
• integration.modules.test_pip.PipModuleTest.test_issue_4805_nested_requirements
• integration.modules.test_pip.PipModuleTest.test_requirements_as_list_of_chains__cwd_not_set__absolute_file_path
• integration.modules.test_pip.PipModuleTest.test_pip_uninstall
• integration.modules.test_pip.PipModuleTest.test_chained_requirements__absolute_file_path
• integration.modules.test_pip.PipModuleTest.test_pip_install_upgrade
• integration.modules.test_pip.PipModuleTest.test_chained_requirements__non_absolute_file_path
• integration.modules.test_pip.PipModuleTest.test_pip_install_multiple_editables_and_pkgs

[Akm0d]

Verified the bug... working on fixing it now

[Akm0d]

Looks like we are getting "Command required for 'pip.install' not found: Could not find a pip binary" rather than the normal dictionary return from a module. Since this return is a string we get the "String indices must be integers" error when ret['stdout'] is accesed

[Akm0d]

This might not be an issue with the test but with salt/modules/pip.py/_get_pip_bin

[waynew]

Iiiiinteresting. I think there are some other failures where pip is missing, too.

[Akm0d]

OK, the actual problem is with virtualenv.create.
On python3 after the testing virtualenv is created, these are the contents of venv/bin: ['python', 'python3']
On python2 with the same tests we get: [u'python3.7', u'activate', u'python', u'activate.xsh', u'pip3.7', u'activate.fish', u'pip3', u'activate_this.py', u'python3', u'easy_install-3.7', u'easy_install', u'wheel', u'activate.ps1', u'python-config', u'activate.csh', u'pip']

pip and such aren't getting added to the virtualenv when we use python3 to run the test suite

[Akm0d]

This is the error from virtualenv_mod.py/create:

{'pid': 1217, 'retcode': 1, 'stdout': "The virtual environment was not created successfully because ensurepip is notnavailable. On Debian/Ubuntu systems, you need to install the python3-venvnpackage using the following command.nn apt-get install python3-venvnnYou may need to use sudo with that command. After installing the python3-venvnpackage, recreate your virtual environment.nnFailing command: ['/tmp/salt-tests-tmpdir/tmp8he8eid8/venv/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']", 'stderr': ''}

[Akm0d]

So... I tried installing python3-venv manually, and there were all kinds of issues with the dependencies (a couple of them needed to be downgraded). But once I got them all worked out and python3-venv installed the tests all passed. There doesn't' seem to be an issue with the salt's base code, maybe something that needs to be modified in kitchen config files?

[waynew]

@s0undt3ch is it in the kitchen config files, or something in the base image? Or would someone else know better?

[s0undt3ch]

The develop branch is not on nox, meaning, the VM is preped at boot by salt-jenkins.
Which requirements files are we talking about?

[Akm0d]

We just need to be sure that python3-venv is installed before the tests run

[Akm0d]

That was it, I added "apt-get install python3-venv" to the provisioner in ubuntu 18.04 in .kitchen.yml and all the tests passed normally. I"m running the full test suite just to be sure then I'll submit a PR

[Akm0d]

This fix works for kitchen, but unrelated tests keep failing so I can't merge :/ Hopefully this will be fixed by nox when we implement it. I"m just going to keep updating the branch on the PR every day until the tests pass and then merge. Does anyone have a better idea? issue closed?

[waynew]

You can mark it with "Pending verification" and/or close it. I expect that your fix is a good'un - that test suite should run when #52687 is merged.

[Akm0d]

This fix needs to be cherry-picked into Neon