Dpl: PyPI deployments not working when "special" chars in password

[ulope]

It seems that some characters (I observed at least $ to cause this) in the PyPI password seem to break things, preventing deployments from working.

The error message when using such a password:

Uploading distributions to https://pypi.python.org/pypi
Uploading hydrachain-0.1.4-py2.py3-none-any.whl
HTTPError: 401 Client Error: You must be identified to edit package information for url: https://pypi.python.org/pypi

Plaintext or encrypted made no difference.

[BanzaiMan]

This is basically the same issue as one that's explained in https://docs.travis-ci.com/user/encryption-keys#Note-on-escaping-certain-symbols. Perhaps it needs to be placed elsewhere?

[ulope]

What I find especially problematic is that this problem also occurs when using the travis cli utility. IMO it should know about the encoding issue and pre-process the value as needed.

[BanzaiMan]

I am not sure if CLI can reliably determine the user _really_ wants the literal FOO=6\&a\(5\!1AB\\ (thus CLI should escape this string as necessary), or it is actually escaped already (so CLI should leave it alone).

Perhaps CLI can add warnings about escaping characters, but that's as far as it can go.

[toomanybrians]

I'm having this same issue. Originally I had an equal sign in my PyPI password, but I changed it to just letters and numbers and also updated and regenerated the encrypted password in the .travis.yml file, but I'm still getting this same error:

creating build/bdist.linux-x86_64/wheel/mpf-0.30.0.dev8.dist-info/WHEEL
Uploading distributions to https://pypi.python.org/pypi
Uploading mpf-0.30.0.dev8-py3-none-any.whl
HTTPError: 401 Client Error: You must be identified to edit package information for url: https://pypi.python.org/pypi

Anyone have any ideas of what else this could be? Could it be a general password error or something?

[danriti]

Running into this error as well:

https://travis-ci.org/danriti/python-traceview/jobs/103205722#L528

The strange thing is that this works when I deploy to the "test" pypi server (https://testpypi.python.org/pypi):

https://travis-ci.org/danriti/python-traceview/jobs/103205213#L514

But for some reason I can't get this to work against production PyPI :cry:

[SimplicityGuy]

I was hitting this issue and was skeptical that removing the = from my password would fix the issue. Low and behold, doing so fixed this.

Please prioritize this fix!

[nedbat]

We just lost a number of hours to this problem as well. I reject the idea that it's the same problem mentioned here: https://docs.travis-ci.com/user/encryption-keys#Note-on-escaping-certain-symbols That note is about the need to escape characters if you use them in a shell command line.

In our case, we were typing the characters into stdin, where there is no need to escape.

[BrunoGrandePhD]

I just ran into this issue as well (my password contained a $). I tried encrypting the password using single-quotes in bash, but that didn't solve the issue. I changed the password to exclude special characters and that worked.

[jakirkham]

Running into this issue on Travis CI. Is the general consensus that we can only have alphanumeric characters in passwords used to deploy to PyPI? That feels a lot less safe IMHO, but maybe we could have the documentation say that at least.

[Tungsteno74]

@danriti I have the exact same problem of your, but i use SemaphoreCi instead of Travis (run well on testpypi but fail on pypi).
Do you have solved the problem?

[rayluo]

As a side note to @danriti :
I was reading your otherwise excellent blog post and everything was promising until I got bitten by this issue. To be fair, it was not your fault but, at a hindsight, you could have mentioned that in your blog too, especially when you've also bitten by it. :-) Still thank you!

[danriti]

@rayluo no problem, glad it was helpful 👍

[rjollos]

I'm having this same issue. Originally I had an equal sign in my PyPI password, but I changed it to just letters and numbers and also updated and regenerated the encrypted password in the .travis.yml file, but I'm still getting this same error:

After many hours of frustration, I found that I need to specify --repo. I assume this is because I was testing by pushing to a remote that was not origin.

[esc]

I have encountred this issue too, it was working fine for a long time for many of my repos, but now I tried for a new repo and it simply refuses to work..

[BanzaiMan]

@esc Do make sure that the secrets are not copied and pasted. Each repository has its own encryption key, so that you'd need to encrypt your secret(s) anew for a new repository.

[esc]

@BanzaiMan yes, I used travis encrypt --repo -- FYI: I have used this successfully for many repos for many years and this breakage is new.

[BanzaiMan]

@esc Are you saying that you escaped the special characters correctly and are still getting incorrect results?

[esc]

I don't have any special characters in my password, only lowercase, uppercase and numbers.

[esc]

Also, in the "preparing deploy" step it says: "Authenticated as esc" so I am assuming it has connected to PyPi correctly?

[esc]

Nevermind, I tried with a bogus password and it still says: "Authenticated as esc".

[esc]

Haha, it even works fine on my old repositories, continuing to debug.

[esc]

Maybe this is because the repo was previously private (earlier today)?

[BanzaiMan]

@esc Ah, that might be it. Currently private and public repository data are separate, so if you change the visibility, you'd need to encrypt the secret anew.

[esc]

I just catted the $HOME/.pypirc file and it turns out the password isn't being decrypted.

after_deploy:
  - cat $HOME/.pypirc

[esc]

I have encrypted that password multiple times today.

[BanzaiMan]

@esc Hmm. That is strange. Which repository are you having problems with?

[esc]

one sec

[esc]

https://github.com/TwentyBN/twentybn-dl

[esc]

I also tried the trick to cat the PyPi password on:

https://github.com/esc/yadoma

and it worked fine. The password is being decrypted and the release is uploaded.

PS: I have already chnaged my PyPi password, so don't try.. ;)

[BanzaiMan]

@esc The secret defined in https://github.com/TwentyBN/twentybn-dl/blob/edefec7390fc0a55638d9e8fdff0482665160fea/.travis.yml#L26 is incorrect.

[esc]

@BanzaiMan ok, re-encrypting now.

[esc]

I am using the command:

travis encrypt --repo TwentyBN/twentybn-dl <PASSWORD>

@BanzaiMan this is correct, right?

[esc]

I re-encrypted and the problem seems to persist.

[esc]

Travis command-line client version 1.8.8.

[esc]

Seems like either the encryption or the decryption or both isn't working correctly.

[esc]

@BanzaiMan also, what do you mean by: "is incorrect"?

[BanzaiMan]

Could you try invoking the command without the password, but supply it in STDIN? After the password, hit return, and ctl-D.

$ travis encrypt -r TwentyBN/twentybn-dl
Reading from stdin, press Ctrl+D when done
foobarbaz
^D
Please add the following to your .travis.yml file:

  secure: "QzWN…"

"Incorrect" means the value cannot be decrypted using the repository's private key.

[esc]

pushed.

[esc]

@BanzaiMan I tried, but to no avail, seems like the travis command line client isn't giving me a good secret.

[BanzaiMan]

@esc. That may indeed be the case. I reckon it may be pulling the public key from .com still. Try adding --org to travis encrypt (in either form).

[esc]

ok, decryption works now, however I still get:

Note: you are uploading to the old upload URL. It's recommended to use the new URL "https://upload.pypi.org/legacy/" or to leave the URL unspecified and allow twine to choose.
Uploading twentybn-dl-163.19.tar.gz
Received "500: Internal Server Error" Package upload appears to have failed.  Retry 1 of 5
Uploading twentybn-dl-163.19.tar.gz
Received "500: Internal Server Error" Package upload appears to have failed.  Retry 2 of 5
Uploading twentybn-dl-163.19.tar.gz
Received "500: Internal Server Error" Package upload appears to have failed.  Retry 3 of 5
Uploading twentybn-dl-163.19.tar.gz
Received "500: Internal Server Error" Package upload appears to have failed.  Retry 4 of 5
Uploading twentybn-dl-163.19.tar.gz
Received "500: Internal Server Error" Package upload appears to have failed.  Retry 5 of 5
It appears you're uploading to pypi.python.org (or testpypi) you've received a 500 error response. PyPI is being phased out for pypi.org. Try using https://upload.pypi.org/legacy/ (or https://test.pypi.org/legacy/) to upload your packages instead. These are the default URLs for Twine now.
HTTPError: 500 Server Error: Internal Server Error for url: https://pypi.python.org/pypi

[esc]

I guess the 401 is solved though, so I'll keep banging my head to solve this next hurdle.

[esc]

Using:

commit 4db4625b163553d89a20a2e2ac8bae73ed838007
Author: Valentin Haenel <[email protected]>
Date:   Wed Jun 21 17:16:35 2017 +0200

    upload to the legacy server

diff --git a/.travis.yml b/.travis.yml
index ae6a3069ac..7effd03372 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -20,6 +20,7 @@ script:
 deploy:
   provider: pypi
   user: esc
+  server: https://upload.pypi.org/legacy/
   distributions: sdist
   skip_cleanup

Solved this problem too.

@BanzaiMan thanks so much for your help!!

[esc]

So yes, the travis command line client might be confused about public and private github repos, using --org or --pro will make it explicit.

$ travis encrypt --help | grep api
    -e, --api-endpoint URL           Travis API server to talk to
        --pro                        short-cut for --api-endpoint 'https://api.travis-ci.com/'
        --org                        short-cut for --api-endpoint 'https://api.travis-ci.org/'

[stale[bot]]

Thanks for contributing to this issue. As it has been 90 days since the last activity, we are automatically closing the issue. This is often because the request was already solved in some way and it just wasn't updated or it's no longer applicable. If that's not the case, please do feel free to either reopen this issue or open a new one. We'll gladly take a look again! You can read more here: https://blog.travis-ci.com/2018-03-09-closing-old-issues

[jessebrennan]

My password contained a # character and I still had this problem. I tried escaping like:

travis encrypt --add deploy.password 'passwordwith\#'

and also

travis encrypt --add deploy.password "passwordwith\\#"

as suggested on the encryption docs. I had success using an alphanumeric password. Am I missing something here?