Warehouse: API Tokens for old repositories too long for Travis CI encrypt

[bryevdv]

Just FYI, the current form of tokens seems to be too long for travis encrypt

(base) ❯ travis encrypt PYPI_TOKEN=pypi:<token>
Detected repository as bokeh/bokeh, is this correct? |yes|
data too large - consider using travis encrypt-file or travis env set

Obviously this is not exactly "your" problem, but I contend that it will seriously hamper the utility of these tokens in a primary use-case scenario. At almost 200 characters, the PyPI tokens are considerably longer than any token I have ever personally encountered on other services. Is it possible to make them shorter?

[di]

Thanks for the report. This looks like a duplicate of #6338 and #6287. Can you try escaping the colon in the token and seeing if this problem remains? e.g.:

-travis encrypt PYPI_TOKEN=pypi:<token>
+travis encrypt PYPI_TOKEN=pypi\:<token>

[bryevdv]

Wow I totally tried to search for existing issues, sorry about that. Escaping the colon did not help, and neither did shortening by removing boilerplate "pypi:" altogether.

[bryevdv]

I should add, it does seem possible to add these as per-project hidden vars in the Travis web UI, I just checked. (I have not actually tested that a token so-added actually works yet) but all things being equal I'd prefer to have these in .travis.yml if possible.

I should add that Travis claims adding hidden env vars in repository settings is suitable for things that "contain sensitive data, such as third-party credentials." so I assume this workaround is safe

[di]

Can you add more details about your platform? Trying this myself, it looks like the total length of the variable name and value must be less than 500 characters:

$ travis encrypt FOO=`python -c "print('a'*497)"`
Please add the following to your .travis.yml file:

  secure: "<encrypted string>"

Pro Tip: You can add it automatically by running with --add.

$ travis encrypt FOO=`python -c "print('a'*498)"`
data too large - consider using travis encrypt-file or travis env set

[bryevdv]

I ran the travis encrypt command on OSX 10.14.6, with travis version 1.8.10

I will note that

travis encrypt FOO=`python -c "print('a'*497)"`

does work

Edit: I swear it did once, but it now no longer does.

[di]

Interesting, I am on a similar platform:

$ sw_vers -productVersion
10.14.5

$ travis --version
1.8.10

Can you try running the same commands I used? Can you see if your token contains any special characters besides the pypi: prefix?

[bryevdv]

There was an underscore. I tried removing it just to see if that made things work, but it did not. The result (without the underscore) appears to not have any other special characters:

>>> token.isalnum()
True

[bryevdv]

OK so this is weird, I started trying the command repeatedly, removing 10 characters from the end, until it worked. It failed until I got to exactly:

travis encrypt "PYPI_TOKEN=<106 characters>

[bryevdv]

OK literally this 107 characters fails:

(base) ❯ travis encrypt "PYPI_TOKEN=22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222221"
data too large - consider using travis encrypt-file or travis env set

but removing the "1" at the end works.

Willing to chalk this up to a bug on Travis side at this point

[di]

What OpenSSL are you using?

$ gem list | grep openssl
openssl (default: 2.1.2)

$ openssl version
LibreSSL 2.6.5

[bryevdv]

~/work/demo.bokeh.org master*
(base) ❯ gem list | grep openssl

~/work/demo.bokeh.org master*
(base) ❯ openssl version
OpenSSL 1.1.1c  28 May 2019

[di]

Seems like this might be due to some weirdness in the underlying openssl library. I'll leave this open for now to see if we get any additional reports.

[foobarna]

Have you tried to check the length of encryption key?

[jaraco]

Have you tried to check the length of encryption key?

How does one do that? I did a search and stumbled on travis-ci/travis.rb#100, which indicates that the key length is 128 bytes. Apparently that was increased to 512 at some point, but there's no mention of that in the ticket.

I've also encountered this issue and discovered a clear disparity in key length between projects.

I have not yet found a way to increase the key length for an existing project with a short key.

[hugovk]

Also ran into this, it definitely looks like a per-project Travis CI problem.

I can travis encrypt for projects where .travis.yml was created after 2017-05-01, but get "data too large" for those created before 2017-04-04:

• https://github.com/travis-ci/travis.rb/issues/687#issuecomment-542248882

[hugovk]

Reported to the Travis CI Community:

• https://travis-ci.community/t/travis-encrypt-data-too-large-for-pypi-tokens-with-older-repos/5792

[pradyunsg]

Thanks for doing that @hugovk! ISTM that we're waiting on a response from the Travis CI folks now. :)

[brainwane]

There's something of a workaround shared in the Travis CI discussion thread, but other than that, seems like we are still waiting for a fix on https://github.com/travis-ci/travis.rb/issues/687 .

[hugovk]

Yes, it's a bit of a tedious workaround, so I've stuck with using a personal token.

Seeing as this has been reported to Travis CI in 2013 and 2014 (https://github.com/travis-ci/travis.rb/issues/100 and https://github.com/travis-ci/travis.rb/issues/221), I'll probably switch to using GitHub Actions to deploy using API tokens with https://github.com/pypa/gh-action-pypi-publish (for example https://github.com/pytest-dev/pytest/pull/6468).

[jaraco]

I’ve been switching to Azure Pipelines for my releases. It’s still a little clunky, and you have to have an Azure account, but one nice feature is the credentials are in a shared store, not stored per project, meaning you can manage the keys/tokens centrally.

[di]

What's the status of this? I haven't seen any discussion about this in a while.

[bryevdv]

I no longer use Travis CI, so I have no updates. From my perspective the issue could be closed.

[hugovk]

I used GitHub Actions for the old repos, and have been using it as first choice for new ones too.

Unfortunately Travis CI don't seen interested, so I marked "Use GHA" as the solution at https://travis-ci.community/t/travis-encrypt-data-too-large-for-pypi-tokens-with-older-repos/5792

[di]

Ok, in that case I'm going to close this as a Travis issue and recommend that people use Actions instead.