Poetry: Unable to publish to Azure Artifacts

[SamEdwardes]

• [x] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Windows 10
• Poetry version: 1.0.9
• Link of a Gist with the contents of your pyproject.toml file: Sorry private repo 👍

Issue

I am trying to publish a python package to my companies Azure DevOps Artifacts. I have built the package using poetry. A run the following at the command line:

❯ poetry --version
Poetry version 1.0.9                                                        
❯ poetry config repositories.azure https://pkgs.dev.azure.com/.../pypi/upload
❯ poetry install
Installing dependencies from lock file

No dependencies to install or update

  - Installing pyia (0.1.0)
❯ poetry build
Building pyia (0.1.0)
 - Building sdist
 - Built pyia-0.1.0.tar.gz

 - Building wheel
 - Built pyia-0.1.0-py3-none-any.whl
❯ poetry publish -r azure --username <> --password <> -vvv

Publishing pyia (0.1.0) to azure 
 - Uploading pyia-0.1.0-py3-none-any.whl 0%
 - Uploading pyia-0.1.0-py3-none-any.whl 100%
 - Uploading pyia-0.1.0-py3-none-any.whl 100%

[UploadError]
HTTP Error 401: Unauthorized

Traceback (most recent call last):
  File "C:\Users\sedwardes\Miniconda3\envs\ia_py\lib\site-packages\clikit\console_application.py", line 131, in run
    status_code = command.handle(parsed_args, io)
  File "C:\Users\sedwardes\Miniconda3\envs\ia_py\lib\site-packages\clikit\api\command\command.py", line 120, in handle
    status_code = self._do_handle(args, io)
  File "C:\Users\sedwardes\Miniconda3\envs\ia_py\lib\site-packages\clikit\api\command\command.py", line 171, in _do_handle
    return getattr(handler, handler_method)(args, io, self)
  File "C:\Users\sedwardes\Miniconda3\envs\ia_py\lib\site-packages\cleo\commands\command.py", line 92, in wrap_handle
    return self.handle()
  File "C:\Users\sedwardes\Miniconda3\envs\ia_py\lib\site-packages\poetry\console\commands\publish.py", line 81, in handle
    client_cert,
  File "C:\Users\sedwardes\Miniconda3\envs\ia_py\lib\site-packages\poetry\masonry\publishing\publisher.py", line 92, in publish
    client_cert=resolved_client_cert,
  File "C:\Users\sedwardes\Miniconda3\envs\ia_py\lib\site-packages\poetry\masonry\publishing\uploader.py", line 110, in upload
    self._upload(session, url)
  File "C:\Users\sedwardes\Miniconda3\envs\ia_py\lib\site-packages\poetry\masonry\publishing\uploader.py", line 205, in _upload
    raise UploadError(e)

I have used Poetry to publish to public repos before. But not Azure or private repos. I have a hunch there may be an issue with Azure Artifacts keyring. The Azure documentation suggests to use twine, and you authenticate with the Azure keyring.

Lastly I tried to copy the code from https://www.nuomiphp.com/eplan/en/104789.html, but I was met with the same error when following this template.

Any ideas on how to publish to Azure Artifacts?

[SamEdwardes]

Quick update. I was able to find a work around. I proceeded with Poetry package development as I normally would. Instead of using Poetry to publish at the end though I used:

> poetry build
> twine upload -r azure dist/*

For this to work I just followed the instructions Azure Artifacts provides for using Twine.

Are there any risks to doing it this way? Seems like it worked for me.

[mxab]

I was able to upload it for our azsure devops server.

We use the upload tokens in the connect feed dialog ( the one under "Upload packages with twine")

Make sure they are in the auth.toml in $HOME/.config/pypoetry/auth.toml

[http-basic]
[http-basic.myfoorepo]
username= ... 
password= ....

then poetry publish -r myfoorepo should work

[SamEdwardes]

Thanks for the feedback @maxab. Our of curiosity were you able to use Poetry with Azure Pipelines?

[mxab]

I'm still a bit confused by Microsofts naming of TFS, Devops and Azure Pipelines. But if you mean the hosted build things, this I haven't tried so only used it in the on-prem installation.

But in general don't you have some kind of secret store there which you then can interpolate during the run.
Something like:

poetry publish -u $(username) -p $(password) ...

[zooba]

Hi, original developer of artifacts-keyring and occasional keyring contributor here 👋

In short, artifacts-keyring wraps up a separate tool that is able to use locally cached Azure DevOps credentials (or pop up a login window) to get a temporary access token to Azure Artifacts. This token isn't tied to the user's username, but comes with the username. To support this in keyring, we added the get_credential API.

It looks like Poetry is currently only using the get_password API (here), which requires a known username. get_credential _may_ accept a username (it's optional), and returns an object that provides both username and password to use.

If the version of keyring you find has get_credential, that's the only one you need to call - backends that don't support it will automatically use the result from get_password. However, if you don't have a username, you can still call get_credential with the index URL to have it generate a valid credential. (It looks like Poetry currently creates its own entry names, which won't work with the artifacts-keyring backend - we need at least the https://dev.azure.com/<name> part of the URL, and fail quickly for any query that doesn't match this.)

There should be no harm in deferring the get_credential call until a 401 has been received, and it's fine to pass the target URL with an always empty username (to avoid looking up any stored credentials). There ought not be a need to specifically search for the presence of the keyring backend.

[ajknzhol]

poetry publish works when used along with personal access token of Azure Artifacts.

export POETRY_REPOSITORIES_AZURE_URL="<FEED URL>"
export POETRY_HTTP_BASIC_AZURE_USERNAME="<Any Thing>"
export POETRY_HTTP_BASIC_AZURE_PASSWORD="<personal access token>"

Note: When using personal access token, username is not required and can be anything

Then, finally, you can upload to Artifacts using the publish command

poetry publish -r azure --build