Packagist: Ability to publish versions outside git tags

[andrewmclagan]

We have git repositories that are Composer, php packages.

To publish a new version of one of these Composer packages on https://packagist.org we currently do the following:

Create a new git tag such as v1.0.0 and push it.
Travis-CI picks up this change and runs our tests.
Packagist (composer) also picks the new tag and auto-magically publishes a new version v1.0.0
The issues we have with this workflow is:

If the tests fail, there a broken version of our package is published.
A PR workflow would alleviate this, although not totally.
Most package mangers such as NPM; have release/publish functionality.
Creating a PR and only merging into master on passing builds is great. Although it still allows maintainers to release versions from master that do not pass CI, if they forego the PR workflow.

[jackbentley]

Correct me if I’m wrong, but this seems fairly trivial.

What you’re describing is more like a pre-release. You’re uncertain of the stability of the release at the point of tagging it.

When using composer, you should really be following Semver - https://semver.org/ - so that anyone depending on your package can update with confidence when a new tag is released.

If you look at section 9 of the Semver doc, it says you can denote a pre-release by tagging it like 1.1.2-alpha etc. It also says a pre-release may be broken.

For an actual release, it states that 1.x.x must always be backwards compatible.

It also says that a version’s contents should never be changed - and it should be released a new version if changes are required.

So your workflow could look something like this: tag it as a pre-release, have it tested and then published on composer all in one go. You could then go further: if your tests pass, you could automate tagging a stable version which would then be published to composer in the same way. This way you know that what you’ve just tagged is stable and you’ll be following Semver correctly.

[andrewmclagan]

IMO semver is not related to waht we are discussing? The only relevant semver issue to this discussion would be what you quoted:

so that anyone depending on your package can update with confidence when a new tag is released.

Package tags represent an immutable version of the package, that should never change. Its very desirable that packages with broken tests should not exist if possible, CI should not be publishing tags, tags should trigger CI.

Most package systems such as NPM to not directly tie a git tag to a published package. They have the ability to directly publish versions e.g. https://docs.npmjs.com/cli/publish

My main issue with having no publish functionality is composer makes the assumption that a semver git tag is a new release of the package that has passed CI checks. This disables one of the core tenants of CI/CD - push out a release through CI and the release process is automated - as a developer I do not have to check if the new tag passes my 1,000s of test cases and can rely on CI to take care of this.

[jackbentley]

I'm not a member of the project team, but I feel like one of the main advantages of having tags and releases the same thing is being able to switch between git clone and a packaged zip at whim. It also allows there to be a direct link from the version to the hosted source code.

Take npm for example, it looks like it allows you to upload a tarball for a release. This tarball may not ever represent anything that's available in the hosted source.

In contrast, Packagist is more like an index of where releases are located - it doesn't store any of the package data, it just links to where to find it.

I personally don't feel like you should be tagging something as a stable version without being sure that the commit is stable. What happens if the tag fails CI and is not stable? You still have a tag in the repo which would then have to be manually deleted. I personally would tag it manually after the commit has passed the CI tests.

Also, I don't think theres anything wrong with tagging in your CI. It's the same as deployment being in your CI. The difference being instead of deploying the release to a machine to publish it, you're tagging the release to be published.

[andrewmclagan]

Hmm. It just feel totally out of sync with every other package ecosystem I have used.

Also there are so many tools, such as git flow that are based around releasing a tag. I just feel CI totally loses it’s relevance if you manually check for passing builds

[sjadema]

I'd like to create my own releases as well. When bundeling assets suchs as CSS / JS, it would be epic to compile those assets in a build step, zip the contents and make a release. One of the reasons i'd like this is to avoid merge conflicts with minified assets. I realise that this would mean that Packagist needs to host its files somewhere (opposed to the repository) so it might be kind of a big change.

[andrewmclagan]

Agreed. I feel this idea won’t float as it requires infrastructure and changes far Byond what’s currently implemented.

[jackbentley]

I think the reasoning behind this not being necessary is that because PHP is an interpreted language, theres no need for anything to be compiled at any point - you run directly from the source code.

In my eyes, I don't think Composer/Packagist is intended for distributing things like CSS or JS - it's specifically targeted at PHP with it's autoloader etc. You should use npm / yarn / etc for things like CSS and JS where you might need Sass > CSS or Babel > JS and the use something like webpack to bring them into your app seamlessly.

[andrewmclagan]

Agreed. More what I was thinking is a way to only register a version against a tag. Not sync all tags, that way ci builds don’t have to be monitored

[alcohol]

Bottom line, ecosystems vary, and feelings and emotions should be left out of it. What we have works and it works well for 99% of all users and use-cases. Lets not complicate things for the few exceptions.