Yii2: Travis - Composer packages can not be fully installed

Created on 13 Nov 2016 · 10Comments · Source: yiisoft/yii2

During working on #12936, I faced with problems related with installing composer packages.

https://github.com/yiisoft/yii2/pull/12936/commits/4ad70760b4f2b0f4baecb69484c23030d3e93446, build.

Failed to clone the [email protected]:jquery/jquery-dist.git repository, try running in interactive mode so that you can enter your GitHub credentials

[RuntimeException]

Failed to execute git clone --mirror '[email protected]:jquery/jquery-dist.git' '/home/travis/.composer/cache/vcs/git-github.com-jquery-jquery-dist.git/'

I found similar issue in Composer Asset Plugin - https://github.com/fxpio/composer-asset-plugin/issues/212.

Then as a workaround I set github-protocols option to https only to force donwload using https. See https://github.com/yiisoft/yii2/pull/12936/commits/4e316baaedb2184e7348cf69d1782a6992c2ef4b. Tests successfully passed, build.
I was testing PHP 5.4 only (for faster builds and to fix specific PHP 5.4 problems), then I decided to test HHVM - https://github.com/yiisoft/yii2/pull/12936/commits/afe44a6f49ca89a395e994a82bd71962851200f6, build.

Reading bower.json of bower-asset/jquery (2.2.4)
GitHub API limit exhausted. Failed to get metadata for the https://github.com/jquery/jquery-dist.git repository, try running in interactive mode so that you can enter your GitHub credentials to increase the API limit

This never happened for me before on Travis. See https://github.com/composer/composer/issues/4884#issuecomment-195229989, seems like it's not required to add auth token anymore.

Then I generated oauth token and added it to composer config as a temporary workaround - https://github.com/yiisoft/yii2/pull/12936/commits/4e6c0e4e5ffe034bc16be1f3cae6c5a3f06c8586, build.

It solved the problem. But if we expand composer update section we will see:

- Installing yiisoft/yii2-composer (2.0.4)

Downloading: Failed

Failed to download yiisoft/yii2-composer from dist: The "https://api.github.com/repos/yiisoft/yii2-composer/zipball/7452fd908a5023b8bb5ea1b123a174ca080de464" file could not be downloaded (HTTP/1.1 401 Unauthorized)

Now trying to download from source

- Installing yiisoft/yii2-composer (2.0.4)

Cloning 7452fd908a5023b8bb5ea1b123a174ca080de464

Then I did some cleanup - https://github.com/yiisoft/yii2/pull/12936/commits/0d9592c6c5ddef3bd9072f27c8b2d0944e58f91f, but tests did not pass again, build.

For example for PHP 5.4 everything is OK, see this build job:

Loading composer repositories with package information

Updating dependencies (including require-dev)

Nothing to install or update

Writing lock file

Generating autoload files

For PHP 7.1 (see this build job) the error appeared again:

Failed to clone the [email protected]:bestiejs/punycode.js.git repository, try running in interactive mode so that you can enter your GitHub credentials

[RuntimeException]

Failed to execute git clone --mirror '[email protected]:bestiejs/punycode.js.git' '/home/travis/.composer/cache/vcs/git-github.com-bestiejs-punycode.js.git/'

I added token again - https://github.com/yiisoft/yii2/pull/12936/commits/c0b31c744e8021a0f46dbc6e248a969db4225b15 and all tests are passing now, build.

But obviously this is insecure and just a temporary workaround.

Why this happens?

If I see recent travis builds, for example https://github.com/yiisoft/yii2/commit/4b1f17db8ff754bef7c5ba1452a443714a888a83 and related job, composer just not installs packages at all:

Loading composer repositories with package information

Updating dependencies (including require-dev)

Nothing to install or update

Writing lock file

Generating autoload files

Why it tries to install everything from scratch in my case and requests auth?

bug

Source

arogachev

Most helpful comment

should be solved by @rob006 PR.

cebe on 1 Dec 2016

🎉2

All 10 comments

I found this article - Adding GitHub token to Travis CI configuration.

I think we should solve this problem in the way that described there.

Also see according Composer docs section - API rate limit and OAuth tokens.

Including non-encrypted Github OAuth token is not only insecure, it causes auto deletion of token, because Github performs a search for it in repositories and automatically deletes it for security reasons.

So even I did everything as described in article, this will fail:

composer config --global github-oauth.github.com "$GITHUB_TOKEN"

with the following exception:

[UnexpectedValueException]

Your github oauth token for github.com contains invalid characters: ""

This is because:

Encrypted variables are not available to untrusted builds such as pull requests coming from another repository.

See Defining encrypted variables in .travis.yml in Travis CI docs.

So someone from core developers needs to commit these changes to .travis.yml file:

env:
  global:
    # moved CUBRID related env variables because token needs to be secure
    - CUBRID_VERSION=9.3.0/CUBRID-9.3.0.0206
    - CUBRID_PDO_VERSION=9.3.0.0001
    - secure: secure_github_oauth_token_generated_with_travis

before_install:
  - composer config -g github-oauth.github.com "$GITHUB_TOKEN"

And the problem should be gone.

I still don't understand why it does not happen (at least once) with the current .travis.yml setup.

Maybe it's related with changing to trusty dist, but in this case why it didn't happen before in #12936 or in #12971?
I thought maybe it's because I committed too often, but this is not the case, because today at the morning it failed again after the first commit.

This problem is currently blocking #12936 from successfully passing tests.

arogachev on 14 Nov 2016

@cebe Did you face with this problem before?

arogachev on 14 Nov 2016

@samdark said that he don't want to include the token in Travis config and has no idea what can cause this error. Other developers, maybe you have some ideas? @photodude maybe you can help?

arogachev on 30 Nov 2016

@arogachev maybe something that will be addressed in the next image update on Travis after 2016-12-01

or more likely an issue with the composer lock file