Auto: Adopting auto in existing project bumps several versions (how to undo mistakes?)

[ericclemmons]

First, this project is great! It's _so_ much better than what I started years ago with https://github.com/ericclemmons/github-semantic-version. 😍

Describe the bug

I'm working to switch from manual releases (with https://github.com/marketplace/actions/release-drafter) to auto in https://github.com/ericclemmons/codelift/pull/58.

With the GitHub finally working (using the lerna release.sh in https://intuit.github.io/auto/pages/getting-started.html#enabling-skip-release-label), my PR went from v0.5.0 to v9.2.1 (https://github.com/ericclemmons/codelift/releases/tag/v9.2.1).

To Reproduce

1. Have a project with existing tags.
2. Run lerna script provided in https://intuit.github.io/auto/pages/getting-started.html#enabling-skip-release-label within a GitHub Action (https://intuit.github.io/auto/pages/build-platforms/github-actions.html).
3. A canary release isn't cut, but instead a real one.

Expected behavior

1. Version would be incremented from what's in lerna.json, or the current version within package.json.

2. There be a way of running a "first time" dry-run so that migrating auto can be safely validated before real tags/releases/packages are published.

   ❯ yarn auto shipit --dry-run
   yarn run v1.19.2
   $ /Users/eric/Projects/ericclemmons/codelift/node_modules/.bin/auto shipit --dry-run
   ⚠  warning   Published canary identifier would be: "-canary.f1bc352"
   ✨  Done in 4.37s.
   

   But in the GitHub Action, yarn auto shipit returns:

   $ auto shipit
   ⚠  warning   NPM: No "NPM_TOKEN" found in environment
   ✔  success   Wrote authentication token string to /home/runner/.npmrc
   ⚠  warning   lerna notice cli v3.20.2
   lerna success found 2 packages
   
   Error: Running command 'npx' with args [lerna, publish, 9.2.2-canary.58.f1bc352.0, --dist-tag, canary, --force-publish, --yes, --no-git-reset, --no-git-tag-version, --exact] failed
   
   
   Changes:
    - codelift: 0.5.0 => 9.2.2-canary.58.f1bc352.0
   

Desktop (please complete the following information):

• OS: macOS
• Browser: Firefox
• Version: 72

Additional context

"auto": "^9.3.1",

[ericclemmons]

auto release --dry-run is a little more helpful than shipit:

~/Projects/ericclemmons/codelift 58-release
❯ yarn auto release --dry-run
yarn run v1.19.2
$ /Users/eric/Projects/ericclemmons/codelift/node_modules/.bin/auto release --dry-run
ℹ  info      Last used release: v0.5.0
⚠  warning   lerna notice cli v3.20.2
lerna success found 1 package

ℹ  info      Using release notes:
...
ℹ  info      Would have released (unless ran with "shipit"): v9.2.1

Now to figure out how it got to v9.2.1 so I can update PRs accordingly or configure something different...

[ericclemmons]

Progress! Deleting the bad tag & removing the release note seems to fix the references in GitHub Actions (despite local --dry-run still seeing v9.2.1):

1. git tag -d v9.2.1
2. git push --delete v9.2.1
3. (Re-run GitHub Action) https://github.com/ericclemmons/codelift/pull/58/checks?check_run_id=408281672

 - codelift: 0.5.0 => 0.5.1-canary.58.45c171c.0

Now to figure out why local (or remote!) was _ever_ coming up with v9.2.1 from v0.5.0 🤔

[hipstersmoothie]

This is awesome that you're using auto! I'll try to debug this with you.

[ericclemmons]

@hipstersmoothie Don't worry about digging in too much right now, I'm still hacking away in https://github.com/ericclemmons/codelift/pull/58 until I hit a brick wall or come out the other side :D

[hipstersmoothie]

This is def a bug on our end. It's because we find your examples, find the one with the highest version, which happens to be next, and use that to find the latest version, which happens to be v9.2.0, which then gets a patch applied to it to get v9.2.1.

[ericclemmons]

Removing version from my private examples' package.json gave a different output:

yarn auto release --dry-run
yarn run v1.19.2
$ /Users/eric/Projects/ericclemmons/codelift/node_modules/.bin/auto release --dry-run
ℹ  info      Last used release: v0.5.0
ℹ  info      Using release notes:
...
ℹ  info      Would have released (unless ran with "shipit"): v0.5.0
✨  Done in 5.88s.

Notice it didn't increment at all 🤔

[ericclemmons]

Getting closer! Going to remove the released plugin and keep chugging along:

$ auto shipit
✖  error     None of the plugins that you are using implement the `canary` command!

"canary" releases are versions that are used solely to test changes. They make sense on some platforms (ex: npm) but not all!

[ericclemmons]

Whoops, I'm a dummy. I missed that plugins overrides npm (I assumed it would stay as a default since it's a dependency)

https://github.com/hipstersmoothie/auto-config-hipstersmoothie/blob/4dc2b725491fa196ac4723fae141f578e79c5382/package.json#L22-L26

[hipstersmoothie]

Notice it didn't increment at all

This is just kinda bad messaging. Incoming PR to make what's happening clearer

[ericclemmons]

First success!!! 🎉

https://github.com/ericclemmons/codelift/pull/58/checks?check_run_id=408308903

I'm cool to keep this open, but we can break off to discuss the following separately:

• Validation before releasing:

  • Documentation update to emphasize --dry-run as part of the Getting Started section. auto release --dry-run has been the most reliable way for me to validate credentials, plugins, etc.. I'm happy to submit this PR if you're interested!

• The 0.5.0 to 9.2.1 behavior:

  • This may be a documentation issue as well: auto release --dry-run would've saved the premature publishing, but perhaps the better answer is to add to Troubleshooting "Remove version from any packages with "private": true"?

• How does canary work with multiple open PRs?

  • For a large project like https://github.com/aws-amplify/amplify-js/, having each commit in a PR available as a package is huge! But with several PRs in flight, I wouldn't instruct users to install @canary, but maybe @PR-123. This could also be a documentation/visibility issue where the GitHub Action/Check could return with a comment or indication how to install the latest (e.g https://zeit.co/github#features)

[hipstersmoothie]

"Remove version from any packages with "private": true"

Done in #894

I'm happy to submit this PR if you're interested!

Please submit a pr! It's valuable to have different points of view.

[hipstersmoothie]

I also wouldn't instruct users to install the canary tag. Publishing to that tag is more just to publish under something other than latest. It acts as a catch all. Having a tag like PR-123 is nice and auto could probably do that, we would just have to clean up the tags after shipit is done. This would probably be a bit more user friendly too.

This could also be a documentation/visibility issue where the GitHub Action/Check could return with a comment or indication how to install the latest

You might not have noticed but this is already happening. Check your PR 😉

https://github.com/ericclemmons/codelift/pull/58#issue-367063354

[hipstersmoothie]

Big heads up though: GitHub actions don't pass secrets to forks at all! So forked PRs will not get canary releases to be published.

[hipstersmoothie]

@ericclemmons you should run yarn install --frozen-lockfile in CI environments.

https://github.com/ericclemmons/codelift/runs/408316662

[hipstersmoothie]

Also [email protected] should have a nicer --dry-run experience

[ericclemmons]

Published PR with canary version: 0.5.1-canary.58.fea0a94.0

Oh, holy crap! It was so subtle I didn't notice it!

What do you think about a PR for it to be a little more visual?

---

📦 Published PR with canary version: 0.5.1-canary.58.fea0a94.0

[hipstersmoothie]

I'm always down for more emoji!

[hipstersmoothie]

Two other plugins you might like are

https://intuit.github.io/auto/plugins/all-contributors/README.html
https://intuit.github.io/auto/plugins/first-time-contributor/README.html

[ericclemmons]

Thanks for your attention tonight @hipstersmoothie. This has been working out great so far!

Like, I'm honestly surprised how well it works! This isn't an easy problem to solve 😅

On the topic of monorepos, I'm looking into alternatives for https://github.com/aws-amplify/amplify-js/:

• You're right: with multiple PRs in flight canary becomes conflicting. The best solution in mind right now would be to have a GitHub Action prune packages that contain PR-123 if PR-123 is not open. (Having a tag like PR-123 to always reference the latest is also handy IMO)

• Amplify uses independent versioning: how does that work with auto?