Poetry: Can't export lockfiles with dependencies that add extras in dev-requirements

[a-recknagel]

• [x] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.

• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Debian GNU/Linux 9 (stretch)

• Poetry version: 1.10
• Link of a Gist with the contents of my pyproject.toml file: here
• Failing command: poetry export -f requirements.txt

Issue

This is a very fresh bug. It's due to 1159148b717cb2a90d872f21a1fe13bd31664e60, in particular this line. As can be seen in the attached pyproject.toml gist, if a dependency is added as a bare in [poetry.dependencies] and with extras in [poetry.dev-dependencies], find_packages will at some point return an empty list. This leads to the following crash when trying to export a lockfile:

# poetry lock
Updating dependencies
Resolving dependencies... (0.3s)

Writing lock file
# poetry export -f requirements.txt -vvv

  Stack trace:

  6  /usr/local/lib/python3.7/site-packages/clikit/console_application.py:131 in run
      129│             parsed_args = resolved_command.args
      130│ 
    → 131│             status_code = command.handle(parsed_args, io)
      132│         except KeyboardInterrupt:
      133│             status_code = 1

  5  /usr/local/lib/python3.7/site-packages/clikit/api/command/command.py:120 in handle
      118│     def handle(self, args, io):  # type: (Args, IO) -> int
      119│         try:
    → 120│             status_code = self._do_handle(args, io)
      121│         except KeyboardInterrupt:
      122│             if io.is_debug():

  4  /usr/local/lib/python3.7/site-packages/clikit/api/command/command.py:171 in _do_handle
      169│         handler_method = self._config.handler_method
      170│ 
    → 171│         return getattr(handler, handler_method)(args, io, self)
      172│ 
      173│     def __repr__(self):  # type: () -> str

  3  /usr/local/lib/python3.7/site-packages/cleo/commands/command.py:92 in wrap_handle
       90│         self._command = command
       91│ 
    →  92│         return self.handle()
       93│ 
       94│     def handle(self):  # type: () -> Optional[int]

  2  /usr/local/lib/python3.7/site-packages/poetry/console/commands/export.py:73 in handle
      71│             dev=self.option("dev"),
      72│             extras=self.option("extras"),
    → 73│             with_credentials=self.option("with-credentials"),
      74│         )
      75│ 

  1  /usr/local/lib/python3.7/site-packages/poetry/utils/exporter.py:43 in export
       41│             dev=dev,
       42│             extras=extras,
    →  43│             with_credentials=with_credentials,
       44│         )
       45│ 

  IndexError

  list index out of range

  at /usr/local/lib/python3.7/site-packages/poetry/utils/exporter.py:76 in _export_requirements_txt
       72│             if not dev
       73│             else self._poetry.package.all_requires,
       74│             with_nested=True,
       75│         ):
    →  76│             package = repository.find_packages(dependency=dependency)[0]
       77│ 
       78│             # If a package is optional and we haven't opted in to it, continue
       79│             if package.optional and package.name not in extra_package_names:
       80│                 continue

Other things I tried:

• a different set of dependencies that had alpha releases (pyparsing v.3.0.2 with its "diagrams" extra) -> same bug
• extras in [poetry.dependencies] and bare dependency in [poetry.dev-dependencies] -> no bug

Pinging @abn, who might know more about this.

[tony]

FYI: If you need to downgrade temporarily while 1.1 stabilizes, see if poetry self update 1.0 works

[a-recknagel]

Thanks for the quick fix, I just tested the new master and can confirm that it works =)

[joshcoales]

I'm getting a slightly different failure when doing poetry lock with poetry 1.1.4, installing pyparsing 3.0.0a2 with diagrams extra, as you mentioned in your issue.

I get this, slightly baffling, error:

Updating dependencies
Resolving dependencies...

  FileNotFoundError

  [Errno 2] No such file or directory: 'C:\\ProgramData\\Anaconda3\\Lib\\venv\\scripts\\nt\\python.exe'

  at C:\ProgramData\Anaconda3\lib\shutil.py:120 in copyfile
       116│
       117│     if not follow_symlinks and os.path.islink(src):
       118│         os.symlink(os.readlink(src), dst)
       119│     else:
    →  120│         with open(src, 'rb') as fsrc:
       121│             with open(dst, 'wb') as fdst:
       122│                 copyfileobj(fsrc, fdst)
       123│     return dst
       124│

Did you check pyparsing+diagrams after this fix?

(My pyproject.toml, for context)

[sinoroc]

@joshcoales

Are you sure it is related to the current topic? What was the command you entered? Can you run the command again with more verbosity (poetry command -vvv ...) and post the output?

Bonus points (maybe it is unrelated):
I recently saw some issues when poetry is running in a _conda_ environment (virtualenv and conda do not really mix so well apparently). Could you maybe try to remove _conda_ (_anaconda_, _miniconda_) from the equation and see if you can still recreate the issue?

[joshcoales]

I'm not certain whether it is the same issue, no. I just saw that this issue was also discussing issues installing pyparsing with the diagrams extra, which is where I am encountering issues.

I've just been doing poetry lock. With the -vvv argument, I get this:
https://gist.github.com/joshcoales/2b77ba660596d094d2e0c0f7f7c0964c

I had some failed attempts to bypass anaconda's python install, I shall have another go

[joshcoales]

@sinoroc

Ah, okay, no issue once I managed to get a non-anaconda python installed. Strange.

[sinoroc]

@joshcoales Thanks for helping with the debug. For information I'll link these other tickets that seem to relate to _virtualenv_ and _conda_.

• https://github.com/python-poetry/poetry/issues/3305
• https://github.com/python-poetry/poetry/issues/3297

If your problem persist, maybe open a new ticket with focus on _virtualenv_ and _conda_. Maybe that is a good lead, maybe it is not. A bit difficult to say at this stage.

[joshcoales]

That's fair enough. It only seemed to happen with pyparsing+diagrams, but yeah. All seems good now, and I don't really use anaconda, so I'm happy to just uninstall that anyway.
Thanks for the help!