Flex: Flex reads composer.json from used bundle and updates flex

[Rubc]

• Installing symfony/flex (v1.3.1): Loading from cache
  Loading composer repositories with package information
  Reading composer.json of jimmycleuren/rrd-bundle (master)
  Updating dependencies (including require-dev)
  Package operations: 208 installs, 1 update, 0 removals
• Updating symfony/flex (v1.3.1 => v1.4.0): Downloading

[stof]

Can you please give more details about the context (which command was running, what were the composer.json and composer.lock before it...)

[kricha]

@stof i think author wants to say, that with flex 1.4.0 composer stuck on downloading dependencies other than flex. I have the same issue on one of my DO droplents (others work fine with the same project)

[nicolas-grekas]

Sorry this makes no sense to me yet. Flex 1.3.1 had an issue: it updated all the dependency graph while maybe only a few packages should have been. But 1.4.0 is free from the issue. Please provide actionable hints.

[kricha]

@nicolas-grekas i'm getting this, when try to run composer install with removed vendor and ~/.composer/cache dirs https://gist.github.com/kricha/f52314f74446ad3834be98e1002aa876

on second run composer install all works fine

[kevinreniers]

Morning, @Rubc and I decided to create this at the end of bd yesterday so the details were a bit lacking, sorry. :smile:

Since updating to flex v1.3 we've noticed the following new behaviour:

• (some) dependencies are being updated during a composer install (i.e. symfony/flex v1.3.1 updated itself to v1.4)
• packages defined in the repositories section are always downloaded/built instead of being fetched from the cache

Because we have to deploy to relatively lightweight machine with few system resources, this is breaking our deploys because they're running out of RAM. They also have very little available disk space, so creating large enough swap files is not an option.

In our build server, we've seen the composer install step of our pipeline increase from taking ~24 seconds to taking over 180 seconds.

Example from one of our target machines:

Installing dependencies (including require-dev) from lock file
Package operations: 210 installs, 0 updates, 0 removals
  - Installing ocramius/package-versions (1.4.0): Loading from cache
  - Installing symfony/flex (v1.3.1): 
Loading from cache
Loading composer repositories with package information
                                                      Reading composer.json of jimmycleuren/rrd-bundle (master)
                                                         
Updating dependencies (including require-dev)

(Then fails, running out of memory)

From our build machine, before v1.3:

Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Package operations: 210 installs, 0 updates, 0 removals
  - Installing ocramius/package-versions (1.4.0): Loading from cache
  - Installing symfony/flex (v1.2.7): Loading from cache
  - ... (more installs here)
Generating autoload files
Executing script security-checker security:check [OK]
Executing script cache:clear [OK]
Executing script assets:install public [OK]

After v1.3.1:

• it updates the dependencies
• it updates flex from 1.3.1 to 1.4
• it processes/downloads the jimmycleuren/rrd-bundle again instead of just loading from the cache
• overwrites the lock file

Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Package operations: 210 installs, 0 updates, 0 removals
  - Installing ocramius/package-versions (1.4.0): Loading from cache
  - Installing symfony/flex (v1.3.1): Loading from cache
Loading composer repositories with package information
������������������������������������������������������                                                      ������������������������������������������������������Reading composer.json of jimmycleuren/rrd-bundle (master)���������������������������������������������������������                                                         ���������������������������������������������������������Updating dependencies (including require-dev)
Package operations: 208 installs, 1 update, 0 removals
  - Updating symfony/flex (v1.3.1 => v1.4.0): Loading from cache
  - Installing alecsammon/php-graphviz (dev-master 8ec6355): Loading from cache
  - Installing symfony/polyfill-ctype (v1.11.0): Loading from cache
  - Installing symfony/polyfill-mbstring (v1.11.0): Loading from cache
  - Installing twig/twig (v2.11.3): Loading from cache
  - ... (more being installed here)
Writing lock file
Generating autoload files
Executing script security-checker security:check [OK]
Executing script cache:clear [OK]
Executing script assets:install public [OK]

After 1.4:

Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Package operations: 210 installs, 0 updates, 0 removals
  - Installing ocramius/package-versions (1.4.0): Loading from cache
  - Installing symfony/flex (v1.4.0): Loading from cache
Loading composer repositories with package information
������������������������������������������������������                                                      ������������������������������������������������������Reading composer.json of jimmycleuren/rrd-bundle (master)���������������������������������������������������������                                                         ���������������������������������������������������������Updating dependencies (including require-dev)
Package operations: 208 installs, 0 updates, 0 removals
  - Installing alecsammon/php-graphviz (dev-master 8ec6355): Loading from cache
  - Installing symfony/polyfill-ctype (v1.11.0): Loading from cache
  - Installing symfony/polyfill-mbstring (v1.11.0): Loading from cache
  - Installing twig/twig (v2.11.3): Loading from cache
  - ... (more installs here)
Writing lock file
Generating autoload files
Executing script security-checker security:check [OK]
Executing script cache:clear [OK]
Executing script assets:install public [OK]

composer.json

{
    "license": "MIT",
    "type": "project",
    "autoload": {
        "psr-4": { "App\\": "src/" }
    },
    "autoload-dev": {
        "psr-4": { "App\\Tests\\": "tests/" }
    },
    "repositories": [
        {
            "type" : "vcs",
            "url" : "[email protected]:jimmycleuren/RrdBundle"
        }
    ],
    "minimum-stability": "dev",
    "prefer-stable": true,
    "require": {
        "php": "^7.2",
        "ext-curl": "*",
        "ext-igbinary": "*",
        "ext-intl": "*",
        "ext-json": "*",
        "ext-memcached": "*",
        "ext-msgpack": "*",
        "ext-newrelic": "*",
        "ext-odbc": "*",
        "ext-pcntl": "*",
        "ext-snmp": "*",
        "ext-soap": "*",
        "ext-sockets": "*",
        "lib-curl": "*",
        "alecsammon/php-graphviz": "dev-master",
        "dapphp/radius": "^2.5",
        "doctrine/doctrine-bundle": "^1.6",
        "doctrine/doctrine-cache-bundle": "^1.2",
        "doctrine/doctrine-migrations-bundle": "^2.0",
        "doctrine/orm": "^2.5",
        "eightpoints/guzzle-bundle": "^7.3",
        "ekino/newrelic-bundle": "^2.0",
        "friendsofsymfony/oauth-server-bundle": "dev-master",
        "friendsofsymfony/rest-bundle": "~2.0",
        "gumlet/php-image-resize": "^1.9",
        "incenteev/composer-parameter-handler": "^2.0",
        "jimmycleuren/rrd-bundle": "dev-master",
        "jms/serializer-bundle": "^3.1",
        "mikey179/vfsstream": "^1.6",
        "nelmio/api-doc-bundle": "^3.0",
        "ob/highcharts-bundle": "1.*",
        "pagerfanta/pagerfanta": "^2.1",
        "php-amqplib/rabbitmq-bundle": "1.*",
        "phpseclib/mcrypt_compat": "^1.0",
        "phpseclib/phpseclib": "~2.0",
        "predis/predis": "^1.1",
        "react/datagram": "~1.0",
        "react/http": "^0.8.3",
        "react/socket": "~1.0",
        "s1lentium/iptools": "^1.1",
        "sensio/framework-extra-bundle": "^5.3",
        "symfony/asset": "^4.3",
        "symfony/console": "^4.3",
        "symfony/dotenv": "^4.3",
        "symfony/event-dispatcher": "^4.3",
        "symfony/filesystem": "^4.3",
        "symfony/finder": "^4.3",
        "symfony/flex": "^1.2",
        "symfony/form": "^4.3",
        "symfony/monolog-bundle": "^3.3",
        "symfony/process": "^4.3",
        "symfony/property-access": "^4.3",
        "symfony/property-info": "^4.3",
        "symfony/security-bundle": "^4.3",
        "symfony/serializer-pack": "^1.0",
        "symfony/swiftmailer-bundle": "^3.2",
        "symfony/templating": "^4.3",
        "symfony/twig-bundle": "^4.3",
        "symfony/validator": "^4.3",
        "twig/extensions": "~1.0",
        "twig/twig": "^1.0||^2.0"
    },
    "require-dev": {
        "ext-sqlite3": "*",
        "doctrine/doctrine-fixtures-bundle": "^3.0",
        "friendsofphp/php-cs-fixer": "^2.13",
        "liip/functional-test-bundle": "~2.0@alpha",
        "phploc/phploc": "^5.0",
        "phpstan/phpstan": "^0.11.5",
        "phpunit/phpunit": "^8.1",
        "rector/rector": "^0.5",
        "roave/security-advisories": "dev-master",
        "sebastian/phpcpd": "^4.1",
        "sensiolabs/security-checker": "^6.0",
        "squizlabs/php_codesniffer": "^3.3",
        "symfony/maker-bundle": "^1.11",
        "symfony/phpunit-bridge": "^4.0",
        "symfony/profiler-pack": "^1.0",
        "symfony/test-pack": "^1.0",
        "symfony/var-dumper": "^4.3"
    },
    "conflict": {
        "symfony/symfony": "*"
    },
    "scripts": {
        "post-install-cmd": [
            "@auto-scripts"
        ],
        "post-update-cmd": [
            "@auto-scripts"
        ],
        "phpstan": "vendor/bin/phpstan analyse src --level=1",
        "auto-scripts": {
            "security-checker security:check": "script",
            "cache:clear": "symfony-cmd",
            "assets:install %PUBLIC_DIR%": "symfony-cmd"
        }
    },
    "config": {
        "preferred-install": {
            "*": "dist"
        },
        "sort-packages": true,
        "process-timeout": 1200
    },
    "extra": {
        "symfony": {
            "allow-contrib": true
        }
    }
}

We're using the built-in composer install steps of php-censor v0.21.0 and deployer v0.0.46.

[nicolas-grekas]

Should be fixed by #526
@kricha I don't get why lines mentioning p-provider-20**.json are repeated so you might want to check after the patch, even on update (with flex already in vendor, and with empty vendor too)

[kricha]

@nicolas-grekas yes, it's fixed with patch. Thanks!