Poetry: publish to devpi

Created on 11 Dec 2018  ·  11Comments  ·  Source: python-poetry/poetry

  • [X] I am on the latest Poetry version.
  • [X] I have searched the issues of this repo and believe that this is not a duplicate.
  • [X] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).
  • OS version and name: Ubuntu Bionic Beaver
  • Poetry version: 0.12.10

Issue


i try to publish to my local devpi server. i added it like this:

poetry config repositories.foo http://localhost:3141/root/dev/

when i try to publish i get his:

poetry publish -r foo -u root -p mypassword --build -vvv                                                                                         12.2s  Di 11 Dez 2018 10:00:55 CET

There are 2 files ready for publishing. Build anyway? (yes/no) [no] yes

Virtualenv my-python-project-py3.7 already exists.
Using virtualenv: /home/preuss/.cache/pypoetry/virtualenvs/my-python-project-py3.7
Building my_python_project (0.0.4)
 - Building sdist
 - Adding: src/my_python_project/__init__.py
 - Adding: src/my_python_project/cli.py
 - Adding: src/my_python_project/core.py
 - Adding: pyproject.toml
 - Adding: README.md
 - Built my_python_project-0.0.4.tar.gz

 - Building wheel
 - Adding: /tmp/tmpcnft47s2/my_python_project-0.0.4/src/my_python_project/__init__.py
 - Adding: /tmp/tmpcnft47s2/my_python_project-0.0.4/src/my_python_project/cli.py
 - Adding: /tmp/tmpcnft47s2/my_python_project-0.0.4/src/my_python_project/core.py
 - Built my_python_project-0.0.4-py3-none-any.whl

Publishing my_python_project (0.0.4) to foo

[RuntimeError]
"my-python-project-0.0.4.tar.gz" does not exist.

Exception trace:
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/cleo/application.py in run() at line 94
   status_code = self.do_run(input_, output_)
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/poetry/console/application.py in do_run() at line 88
   return super(Application, self).do_run(i, o)
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/cleo/application.py in do_run() at line 197
   status_code = command.run(input_, output_)
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/poetry/console/commands/command.py in run() at line 77
   return super(BaseCommand, self).run(i, o)
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/cleo/commands/base_command.py in run() at line 146
   status_code = self.execute(input_, output_)
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/cleo/commands/command.py in execute() at line 107
   return self.handle()
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/poetry/console/commands/publish.py in handle() at line 54
   self.option("repository"), self.option("username"), self.option("password")
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/poetry/masonry/publishing/publisher.py in publish() at line 84
   return self._uploader.upload(url)
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/poetry/masonry/publishing/uploader.py in upload() at line 97
   self._upload(session, url)
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/poetry/masonry/publishing/uploader.py in _upload() at line 193
   self._register(session, url)
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/poetry/masonry/publishing/uploader.py in _register() at line 258
   raise RuntimeError('"{0}" does not exist.'.format(file.name))

publish [-r|--repository REPOSITORY] [-u|--username USERNAME] [-p|--password PASSWORD] [--build]

the devpi logs this:

2018-12-11 09:01:01,089 INFO  [req44] POST /root/dev/
2018-12-11 09:01:01,193 ERROR [req44] [Wtx6] while handling http://localhost:3141/root/dev/:
no project named 'my-python-project' was ever registered
Bug Packaging stale
Source
picture xsteadfastx
👍1

Most helpful comment

Does this really need closing? Has it been fixed?

picture kierun on 2 Dec 2019
👍6

All 11 comments

i investigated a little more. it turns out there is a problem with underscores. if my project is named without underscores, everything gets published well.

xsteadfastx picture xsteadfastx on 12 Dec 2018

i investigated a little more. it turns out there is a problem with underscores. if my project is named without underscores, everything gets published well.

xsteadfastx picture xsteadfastx on 13 Dec 2018

I have very similar problem with publishing package named ttr.run.

Full test of publishing a package to devpi-server

Before experimenting, login to devpi-server:

$ devpi login javl
...

Make sure the package ttr.run is not present in the devpi index

$ devpi remove ttr.run
*redirected: http://127.0.0.1:3141/javl/dev/ttr-run
GET http://127.0.0.1:3141/javl/dev/ttr-run
404 Not Found: no project 'ttr-run'

This is fine, it is not there.

Build and try to publish by poetry

Build the package by poetry:

$ poetry build

and publish it by poetry:

$ poetry publish -r local -u javl -p verysecreteone

Publishing ttr.run (0.3.0) to local
 - Uploading ttr.run-0.3.0-py3-none-any.whl 100%
 - Uploading ttr.run-0.3.0.tar.gz 100%

According to responses, one would call it a success

Check the result (of poetry publishing)

devpi-server logs

Let's check devpi-server logs

$ devpi-server --log
......

show records (following deletion):

2019-04-12 10:57:23,110 INFO  [req114] GET /javl/dev/ttr.run/
2019-04-12 10:57:23,115 INFO  [req115] GET /javl/dev/ttr-run
2019-04-12 10:57:34,283 INFO  [req116] POST /javl/dev/+simple/
2019-04-12 10:57:34,284 INFO  [req116] starting +simple
2019-04-12 10:57:34,679 INFO  [req117] POST /javl/dev/+simple/
2019-04-12 10:57:34,680 INFO  [req117] starting +simple
2019-04-12 10:58:33,919 INFO  [req118] GET /javl/dev/ttr.run/
2019-04-12 10:58:33,929 INFO  [req119] GET /javl/dev/ttr-run
2019-04-12 10:58:50,417 INFO  [req120] POST /javl/dev/+simple/
2019-04-12 10:58:50,418 INFO  [req120] starting +simple
2019-04-12 10:58:50,803 INFO  [req121] POST /javl/dev/+simple/
2019-04-12 10:58:50,804 INFO  [req121] starting +simple

All looks fine.

devpi client listing packages (package missing)

Now use devpi client to check, if the packages are present in the index:

$ devpi list ttr.run
*redirected: http://127.0.0.1:3141/javl/dev/ttr-run
GET http://127.0.0.1:3141/javl/dev/ttr-run
404 Not Found: no project 'ttr-run'

Upload packages by devpi client (works)

$ cd dist
$ devpi upload ttr.run-0.3.0*
register ttr.run-0.3.0 to http://127.0.0.1:3141/javl/dev/
file_upload of ttr.run-0.3.0-py3-none-any.whl to http://127.0.0.1:3141/javl/dev/
register ttr.run-0.3.0 to http://127.0.0.1:3141/javl/dev/
file_upload of ttr.run-0.3.0.tar.gz to http://127.0.0.1:3141/javl/dev/

and check, if it is present

$ devpi list ttr.run
*redirected: http://127.0.0.1:3141/javl/dev/ttr-run
http://127.0.0.1:3141/javl/dev/+f/64f/72bb608ab8fbd/ttr.run-0.3.0-py3-none-any.whl
http://127.0.0.1:3141/javl/dev/+f/c69/5743f17f1b724/ttr.run-0.3.0.tar.gz

Success in devpi-server logs

How does the success looks like in devpi-server logs:

Upload records:

2019-04-12 11:13:18,473 INFO  [req127] POST /javl/dev/
2019-04-12 11:13:18,475 INFO  [req127] [Wtx2564] set_metadata ttr-run-0.3.0
2019-04-12 11:13:18,476 INFO  [req127] javl/dev: got submit release info 'ttr.run'
2019-04-12 11:13:18,483 INFO  [req127] [Wtx2564] fswriter2565: committed: keys: 'javl/dev/ttr-run/.versions','javl/dev/ttr-run/0.3.0/.config','javl/dev/.projects'
2019-04-12 11:13:18,499 INFO  [req128] POST /javl/dev/
2019-04-12 11:13:18,573 INFO  [req128] [Wtx2565] added 'releasefile' link javl/dev/+f/64f/72bb608ab8fbd/ttr.run-0.3.0-py3-none-any.whl
2019-04-12 11:13:18,574 INFO  [req128] [Wtx2565] set_metadata ttr-run-0.3.0
2019-04-12 11:13:18,590 INFO  [NOTI] [Rtx2565] Committing 6 new documents to search index.
2019-04-12 11:13:18,598 INFO  [req128] [Wtx2565] fswriter2566: committed: keys: 'javl/dev/+f/64f/72bb608ab8fbd/ttr.run-0.3.0-py3-none-any.whl','javl/dev/ttr-run/.simple','javl/dev/ttr-run/0.3.0/.config', files_commit: +files/javl/dev/+f/64f/72bb608ab8fbd/ttr.run-0.3.0-py3-none-any.whl
2019-04-12 11:13:18,603 INFO  [req129] POST /javl/dev/
2019-04-12 11:13:18,606 INFO  [req129] [Wtx2566] set_metadata ttr-run-0.3.0
2019-04-12 11:13:18,606 INFO  [req129] javl/dev: got submit release info 'ttr.run'
2019-04-12 11:13:18,614 INFO  [req129] [Wtx2566] fswriter2567: committed: keys: 'javl/dev/ttr-run/0.3.0/.config'
2019-04-12 11:13:18,619 INFO  [req130] POST /javl/dev/
2019-04-12 11:13:18,626 INFO  [req130] [Wtx2567] added 'releasefile' link javl/dev/+f/c69/5743f17f1b724/ttr.run-0.3.0.tar.gz
2019-04-12 11:13:18,626 INFO  [req130] [Wtx2567] set_metadata ttr-run-0.3.0
2019-04-12 11:13:18,638 INFO  [req130] [Wtx2567] fswriter2568: committed: keys: 'javl/dev/ttr-run/.simple','javl/dev/+f/c69/5743f17f1b724/ttr.run-0.3.0.tar.gz','javl/dev/ttr-run/0.3.0/.config', files_commit: +files/javl/dev/+f/c69/5743f17f1b724/ttr.run-0.3.0.tar.gz
2019-04-12 11:13:18,685 INFO  [NOTI] [Rtx2565] Finished committing 6 documents to search index.
2019-04-12 11:13:18,796 INFO  [NOTI] [Rtx2566] Committing 6 new documents to search index.
2019-04-12 11:13:18,849 INFO  [NOTI] [Rtx2566] Finished committing 6 documents to search index.
2019-04-12 11:13:18,937 INFO  [NOTI] [Rtx2567] Committing 5 new documents to search index.
2019-04-12 11:13:19,159 INFO  [NOTI] [Rtx2567] Finished committing 5 documents to search index.
2019-04-12 11:13:19,243 INFO  [NOTI] [Rtx2568] Committing 5 new documents to search index.
2019-04-12 11:13:19,260 INFO  [NOTI] [Rtx2568] Finished committing 5 documents to search index.

and traces of listing:

2019-04-12 11:13:24,743 INFO  [req131] GET /javl/dev/ttr.run/
2019-04-12 11:13:24,747 INFO  [req132] GET /javl/dev/ttr-run

Versions

Regarding version:

$ devpi --version
devpi-client 4.3.0

current devpi server: http://127.0.0.1:3141/
    devpi-server 4.8.1
    devpi-web 3.5.0
$ poetry --version
Poetry 0.12.11

Conclusions

  • devpi-server logs differ for publishing via poetry publish (much simpler) and by devpi client.
  • apparently, there is something related to naming packages, see ttr.run being redirected to ttr-run in devpi-server logs
  • naming issues do not relate only to underscores, but also to dots in package names.
  • the issue could be with poetry or with devpi-server, next action on resolving this issue would be to investigate this.
  • there is an workaround: build package by poetry and then use devpi client to upload it
vlcinsky picture vlcinsky on 12 Apr 2019
👍1

Since version 5.0, devpi doesn't require package registration anymore, so poetry publish should work once you update your devpi-server installation.

hoefling picture hoefling on 6 Sep 2019

It seems that I'm encountering the same problem with devpi-server==5.1.0. And my package doesn't even contain any dashes or underscores in its name.

stinovlas picture stinovlas on 23 Sep 2019

@stinovlas I can't confirm that with devpi-server==5.1.0. What's your server log on poetry publish? Here's a recent example from one of mine instances, publishing .tar.gz, .whl and docs:

2019-09-23 16:38:16,273 INFO  [req25421] POST /company/dev/
2019-09-23 16:38:16,393 INFO  [req25421] [Wtx4429] set_metadata samplepkg-sphinxcontrib-0.1.0
2019-09-23 16:38:16,393 INFO  [req25421] company/dev: got submit release info 'samplepkg-sphinxcontrib'
2019-09-23 16:38:16,395 INFO  [req25421] [Wtx4429] added 'releasefile' link company/dev/+f/ec1/f13e044d8339a/samplepkg-sphinxcontrib-0.1.0.tar.gz
2019-09-23 16:38:16,395 INFO  [req25421] [Wtx4429] set_metadata samplepkg-sphinxcontrib-0.1.0
2019-09-23 16:38:16,518 INFO  [req25421] [Wtx4429] fswriter4430: committed: keys: 'company/dev/samplepkg-sphinxcontrib/.simple','company/dev/samplepkg-sphinxcontrib/0.1.0/.config','company/dev/.projects','company/dev/samplepkg-sphinxcontrib/.versions','company/dev/+f/ec1/f13e044d8339a/samplepkg-sphinxcontrib-0.1.0.tar.gz', files_commit: +files/company/dev/+f/ec1/f13e044d8339a/samplepkg-sphinxcontrib-0.1.0.tar.gz
2019-09-23 16:38:16,537 INFO  [req25422] POST /company/dev/
2019-09-23 16:38:17,089 INFO  [req25422] [Wtx4430] not re-registering same metadata for samplepkg-sphinxcontrib-0.1.0
2019-09-23 16:38:17,096 INFO  [req25422] company/dev: got submit release info 'samplepkg-sphinxcontrib'
2019-09-23 16:38:17,103 INFO  [NOTI] [Rtx4430] Committing 5 new documents to search index.
2019-09-23 16:38:17,105 INFO  [req25422] [Wtx4430] added 'releasefile' link company/dev/+f/c7b/0307e35270867/samplepkg_sphinxcontrib-0.1.0-py35-none-any.whl
2019-09-23 16:38:17,106 INFO  [req25422] [Wtx4430] set_metadata samplepkg-sphinxcontrib-0.1.0
2019-09-23 16:38:17,190 INFO  [NOTI] [Rtx4430] Finished committing 5 documents to search index.
2019-09-23 16:38:17,212 INFO  [req25422] [Wtx4430] fswriter4431: committed: keys: 'company/dev/samplepkg-sphinxcontrib/.simple','company/dev/samplepkg-sphinxcontrib/0.1.0/.config','company/dev/+f/c7b/0307e35270867/samplepkg_sphinxcontrib-0.1.0-py35-none-any.whl', files_commit: +files/company/dev/+f/c7b/0307e35270867/samplepkg_sphinxcontrib-0.1.0-py35-none-any.whl
2019-09-23 16:38:17,395 INFO  [NOTI] [Rtx4431] Committing 5 new documents to search index.
2019-09-23 16:38:17,469 INFO  [NOTI] [Rtx4431] Finished committing 5 documents to search index.
2019-09-23 16:38:19,088 INFO  [req25423] GET /company/dev/+api
2019-09-23 16:38:19,320 INFO  [req25424] POST /+login
2019-09-23 16:38:19,771 INFO  [req25425] POST /company/dev/
2019-09-23 16:38:19,779 INFO  [req25425] [Wtx4431] set_metadata samplepkg-sphinxcontrib-0.1.0
2019-09-23 16:38:19,779 INFO  [req25425] company/dev: got submit release info 'samplepkg-sphinxcontrib'
2019-09-23 16:38:19,781 INFO  [req25425] [Wtx4431] added 'doczip' link company/dev/+f/495/b4f4b48ca5f76/samplepkg-sphinxcontrib-0.1.0.doc.zip
2019-09-23 16:38:19,781 INFO  [req25425] [Wtx4431] set_metadata samplepkg-sphinxcontrib-0.1.0
2019-09-23 16:38:19,785 INFO  [req25425] [Wtx4431] fswriter4432: committed: keys: 'company/dev/+f/495/b4f4b48ca5f76/samplepkg-sphinxcontrib-0.1.0.doc.zip','company/dev/samplepkg-sphinxcontrib/0.1.0/.config', files_commit: +files/company/dev/+f/495/b4f4b48ca5f76/samplepkg-sphinxcontrib-0.1.0.doc.zip
2019-09-23 16:38:19,992 INFO  [NOTI] [Rtx4432] Committing 11 new documents to search index.
2019-09-23 16:38:20,124 INFO  [NOTI] [Rtx4432] Finished committing 11 documents to search index.
2019-09-23 16:38:20,308 INFO  [NOTI] [Rtx4432] Committing 11 new documents to search index.
2019-09-23 16:38:20,696 INFO  [NOTI] [Rtx4432] Finished committing 11 documents to search index.
2019-09-23 16:38:53,859 INFO  [req25426] GET /company/dev
hoefling picture hoefling on 23 Sep 2019

Yesterday I got exactly the same results as vlcinsky did, but I wasn't able to recreate it with local devpi and minimal working example. However, I have run into a different error. Now, the upload does not start in a first place. Meaning that there is no devpi-server log and poetry is stuck in this state and eventualy crashes with ConnectionError:

Publishing my_package (0.2.0) to test
 - Uploading my_package-0.2.0-py3-none-any.whl 0%                                                
[ConnectionError]                
('Connection aborted.', OSError(0, 'Error'))  

Exception trace:
 /home/jmusilek/.poetry/lib/poetry/_vendor/py3.7/cleo/application.py in run() at line 94
   status_code = self.do_run(input_, output_)
 /home/jmusilek/.poetry/lib/poetry/console/application.py in do_run() at line 88
   return super(Application, self).do_run(i, o)
 /home/jmusilek/.poetry/lib/poetry/_vendor/py3.7/cleo/application.py in do_run() at line 197
   status_code = command.run(input_, output_)
 /home/jmusilek/.poetry/lib/poetry/console/commands/command.py in run() at line 77
   return super(BaseCommand, self).run(i, o)
 /home/jmusilek/.poetry/lib/poetry/_vendor/py3.7/cleo/commands/base_command.py in run() at line 146
   status_code = self.execute(input_, output_)
 /home/jmusilek/.poetry/lib/poetry/_vendor/py3.7/cleo/commands/command.py in execute() at line 107
   return self.handle()
 /home/jmusilek/.poetry/lib/poetry/console/commands/publish.py in handle() at line 54
   self.option("repository"), self.option("username"), self.option("password")
 /home/jmusilek/.poetry/lib/poetry/masonry/publishing/publisher.py in publish() at line 84
   return self._uploader.upload(url)
 /home/jmusilek/.poetry/lib/poetry/masonry/publishing/uploader.py in upload() at line 101
   self._upload(session, url)
 /home/jmusilek/.poetry/lib/poetry/masonry/publishing/uploader.py in _upload() at line 185
   self._do_upload(session, url)
 /home/jmusilek/.poetry/lib/poetry/masonry/publishing/uploader.py in _do_upload() at line 202
   resp = self._upload_file(session, url, file)
 /home/jmusilek/.poetry/lib/poetry/masonry/publishing/uploader.py in _upload_file() at line 237
   headers={"Content-Type": monitor.content_type},
 /home/jmusilek/.poetry/lib/poetry/_vendor/py3.7/requests/sessions.py in post() at line 581
   return self.request('POST', url, data=data, json=json, **kwargs)
 /home/jmusilek/.poetry/lib/poetry/_vendor/py3.7/requests/sessions.py in request() at line 533
   resp = self.send(prep, **send_kwargs)
 /home/jmusilek/.poetry/lib/poetry/_vendor/py3.7/requests/sessions.py in send() at line 646
   r = adapter.send(request, **kwargs)
 /home/jmusilek/.poetry/lib/poetry/_vendor/py3.7/requests/adapters.py in send() at line 498
   raise ConnectionError(err, request=request)

publish [-r|--repository REPOSITORY] [-u|--username USERNAME] [-p|--password PASSWORD] [--build]

Uploading package through devpi upload still works fine.

Output of devpi --version:

devpi-client 5.0.0

current devpi server: http://localhost:3141/
    devpi-server 5.1.0

I tried both poetry 0.12.17 and 1.0.0b1.

stinovlas picture stinovlas on 24 Sep 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] picture stale[bot] on 23 Nov 2019

Closing this issue automatically because it has not had any activity since it has been marked as stale. If you think it is still relevant and should be addressed, feel free to open a new one.

stale[bot] picture stale[bot] on 30 Nov 2019

Does this really need closing? Has it been fixed?

kierun picture kierun on 2 Dec 2019
👍6

Yes I believe it has been fixed. I managed to get it working thanks to a few links below.

What I did:
1) Connect to DevPi Server: devpi use http://localhost:3141
2) Log in: devpi login root
1) Create non-root user: devpi user -c packages [email protected] password=packages
2) Create new package index: devpi index -c packages/stable bases=root/pypi volatile=False
3) Add the Devpi repository to poetry: poetry config repositories.stable https://localhost:3141/packages/stable
4) Publish to Devpi: poetry publish -r stable -u ${username} -p {password}

Then it should work. If you setup you've set the devpi repo as the default source, you can then just run poetry add <package> and it should work.

Links:
1) https://github.com/python-poetry/poetry/issues/742#issuecomment-457740940
2) https://stefan.sofa-rockers.org/2017/11/09/getting-started-with-devpi/
3) https://python-poetry.org/docs/repositories/#using-a-private-repository

VikashKothary picture VikashKothary on 12 Mar 2020
👍3
Was this page helpful?
0 / 5 - 0 ratings

Related issues

alexlatchford picture alexlatchford  ·  3Comments
jeremy886 picture jeremy886  ·  3Comments
sobolevn picture sobolevn  ·  3Comments
thejohnfreeman picture thejohnfreeman  ·  3Comments
thmo picture thmo  ·  3Comments