Silverstripe-framework: RFC: Exclude tests & docs from dist packages

Created on 8 May 2017 · 2Comments · Source: silverstripe/silverstripe-framework

Overview

Ignoring .git, this is a breakdwon

 96K    _config
4.0K    _config.php
4.0K    _register_database.php
8.0K    cli-script.php
 36K    client
4.0K    composer.json
4.0K    CONTRIBUTING.md
8.7M    docs
632K    lang
4.0K    LICENSE
4.0K    main.css
 12K    main.php
4.0K    main.php5
4.0K    phpunit.xml.dist
4.0K    README.md
4.0K    sake
  0B    silverstripe_version
4.1M    src
272K    templates
4.7M    tests
360K    thirdparty
4.0K    web.config
 19M    total

70% of the files come from these two items, neither of which are necessary to run SilverStripe, nor to develop with it.

8.7M docs
4.7M tests

Furthermore, it encourages the practise of splitting heavyweight tests into other modules. It would be good to consider, for example, whether frameworktest should be test fixtures in the framework module. But forcing every user to download this module seems inappropriate. The underlying issue that i that

Documentation is arguably of assistance to developers, but I question whether reading local markdown files from framework/docs/en/ is a sufficiently superior experience to reading it on doc.silverstripe.org or github.com that we should double the download size of every user in order to provide it.

Acceptance Criteria

Exclude all doc sources from core package via gitattributes
Exclude all test sources from core package via gitattributes
Ensure that combined test builds (such as what we're planning in installer) run a --prefer-source install
Ensure that our documentation generator runs a --prefer source install
Silverstripe.org/download follows these practices (does not contain extraneous files)

Alternatives

The most common counter-proposal as been a post-install script that deletes tests and/or docs, although that only solves disk size, not download size, and it's an order-of-magnitude more difficult (compared to a few lines in .gitattributes)

External discussion

Some discussion of these idea around the internet; it's controversial and it looks like Symfony started doing this but changed their mind. Seldaek (composer lead) initially spoke in support of leaving in tests and then later pulled back to a "there are arguments to both approaches, I don't want to promote one approach over the other"

Pull requests

affectv4 changminor efforeasy impaclow rfaccepted

Source

sminnee

👍2

Most helpful comment

+1 - frankly I thought this was already in place. We're already removing those folders from the archives created for silverstripe.org/download by the silverstripe/cow tool. So this is really only about projects using composer install, which in some situations uses "dist" files (zip archives). These are automatically created by github when requested, and cached for fast access later on.

Overall, this will reduce the average time a composer install takes for most use cases (where --prefer-dist is viable), particularly when bandwidth is a concern (such as when downloading a bunch of github archives hosted in the US when you're in NZ or EU).

Regarding docs use, I always found doc.silverstripe.org too slow and cumbersome to look something up in the docs - compared to just autocompleting a markdown file path in my editor. But then again, I'm probably not the average SilverStripe dev in this regard :D And that use case will continue to work because I'm generally not using --prefer-dist for OSS dev.

chillu on 9 May 2017

👍3

All 2 comments

Ensure that combined test builds (such as what we're planning in installer) run a --prefer-source install

Is there a way of installing with --prefer-dist first, and then re-installing only the modules we need with --prefer-source? I'd rather not git clone all symfony modules we install as dependencies.