Azure-pipelines-tasks: TwineAuthenticateV1 username set incorrectly when using Authentication Token

[nocarryr]

Note

Issues in this repo are for tracking bugs, feature requests and questions for the tasks in this repo

For a list:
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks

If you have an issue or request for the Azure Pipelines service, use developer community instead:

https://developercommunity.visualstudio.com/spaces/21/index.html )

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Bug

Enter Task Name: TwineAuthenticateV1

list here (V# not needed):
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks

Environment

• Server - Azure Pipelines

  • If using Azure Pipelines, provide the account name, team project name, build definition name/build number:
    
    
    
    • https://dev.azure.com/matt0479/matt
    
    
    • Builds
      
      
      
      
      
      • 20191108.15
      
      
      
      • 20191108.16
      
      
      
      • 20191108.17
      
      
      
    
    

• Agent - Hosted:

  • If using Hosted agent, provide agent queue name: Unsure if relevant

Issue Description

When using the "Python Package Upload" Service Connection with the Authentication Token option (PyPI API Token), the current task's implementation sets the value of the username to "build" in the generated pypirc config:

        case "token":
            const token = externalAuth.parameters["apitoken"];
            tl.debug(tl.loc("Info_AddingTokenAuthEntry", feedUri));
            externalAuthArray.push(new AuthInfo({
                    feedName: endpointName,
                    feedUri,
                    isInternalSource: false,
                } as IPackageSource,
                AuthType.Token,
                "build", // fake username, could be anything.
                token,
                ));
            break;

Per the PyPI documentation, when using an API token, the username is specific and should be set to __token__.

To verify this, I added a script in my pipeline to replace the username values accordingly and was able to properly authenticate:

Replacement Script

Working Upload to PyPI After Script

[eduardostarling]

I am facing the same issue, resolved it slightly differently by running the following before twine upload:
sed -i 's/username=build/username=__token__/g' $(PYPIRC_PATH)

There is even a test case validating the "build" username: https://github.com/microsoft/azure-pipelines-tasks/blob/master/Tasks/TwineAuthenticateV1/Tests/L0.ts

[nocarryr]

@eduardostarling your workaround is way cleaner than mine. Mind if I steal it?

[shubham90]

@nocarryr @eduardostarling You should be able to add the Auth token and username for pypi.org in the service connection if you choose the "Username and password" radio button. The API token option is available only for Azure DevOps feeds and therefore the default username is "build". You can either use a script to change the username(not recommended) or use the radio button button option to add your own username and password(recommended).

For any external endpoint, we recommend using the username and password option. I verified that it works with pypi.org if I set __token__ as username and auth token as password. Please let me know if it doesn't work for you.

[m01]

This is also an issue when uploading Python packages to Artifactory when using access tokens for authentication.

Using username/password authentication as suggested by @shubham90 does not work - it gives the "Enter a valid Password" error. If I enter abc as password it's accepted, so I guess Azure DevOps' Service Connection UI is unhappy about the length of the token (JWT)?

With Artifactory the issue actual result/issue is subtly different: When using access tokens with HTTP basic auth, as twine does, you need to:

[...] access Artifactory using the same user name provided when creating the token (with -d "username=<USERNAME>").

For example, to use an access token as a bearer token to ping Artifactory you could use:
curl -u<USERNAME>:<TOKEN> http://JFROG_PLATFORM_URL/router/api/v1/system/ping

source.

In practice, this means that you end up with errors like this in Azure DevOps build logs:

Uploading abcd.ef.blah-mytag-py2.py3-none-any.whl

  0%|          | 0.00/12.3k [00:00<?, ?B/s]
100%|██████████| 12.3k/12.3k [00:00<00:00, 907kB/s]HTTPError: 401 Client Error: Unauthorized for url: https://artifactory/api/pypi/pypi-local

Content received from server:
{
  "errors" : [ {
    "status" : 401,
    "message" : "Token principal mismatch."
  } ]
}
ERROR: InvocationError: '/opt/agent/_work/1/s/.venv/bin/python -m twine upload --verbose -r blah --config-file /opt/agent/_work/_temp/twineAuthenticate/AbCd1/.pypirc dist/*'

and in the Artifactory logs:

2020-04-24T12:34:56.123Z [jfrt ] [ERROR] [123456abcdef1234] [okenAuthenticationProvider:208] [http-nio-8081-exec-4] - Principal mismatch for token with id 'abcde123-4567-abcd-1234-abcdef123456'.

The sed trick suggested by @eduardostarling in the comment further above does work.

Other than from @shubham90's comment, it's not obvious to me that "Token" authentication is only supported for Azure DevOps feeds. I checked the Twine upload service connection docs (the "Learn more" link there even explicitly mentions "[...] Use PATs for non-Microsoft tools integrated with Azure DevOps but that don't support Microsoft account or Azure AD authentication. Examples include Git, NuGet, or Xcode. [...]", emphasis mine) and the TwineAuthenticate docs.