Lerna: Canary publish results in null-alpha.200+925ed27

[kirill-konshin]

I am trying to publish a canary for not-yet-released package.

When I run

$ npx lerna publish --canary --force-publish

I get following:

Found 3 packages to publish:
 - @xxx/aaa => null-alpha.200+925ed27
 - @xxx/bbb => null-alpha.200+925ed27
 - @xxx/ccc => null-alpha.200+925ed27

My lerna.json has version:

{
  "lerna": "3.4.3",
  "packages": [
    "aaa",
    "bbb",
    "ccc"
  ],
  "version": "4.0.0"
}

My root package.json has same version 4.0.0 as well as all other package.jsons.

[kirill-konshin]

It seems that describeRef call tries

        describeRef({
          match: `${tagVersionPrefix}*.*.*`,
          cwd: this.execOpts.cwd,
        })

Tries to find prev version, and fails, since there are no matching tags yet.

Moreover, if the previous tags do exist, but with different --tag-version-prefix situation is the same.

I wonder why canary tries to get PREVIOUS version, if there is an explicit version set in lerna.json or package.json, am I missing something?

How should I make a canary release of upcoming version then?

[legenddcr]

My CI script works well on lerna 2.x but failed on the latest release 3.4.3.

lerna publish --canary=xyz.123 --yes --no-git-tag-version --no-push

similar null prefix is reported. --yes failed to disable prompt which works well in lerna 2.x.

[evocateur]

--canary no longer accepts an argument in lerna v3. Canary never tags or pushes git changes.

Is your CI clone shallow, by any chance? If so, it's likely not deep enough to encompass the v4.0.0 annotated tag.

[kirill-konshin]

Let me clarify. There's no 4.0.0 yet in git. There's no even 4.0.0-alpha.0. But I was going to start publishing the canary releases for upcoming 4.0.0. Is there a way to do that?

[evocateur]

As long as there are reachable v3.x.x tags:

npx lerna publish premajor --canary

[kirill-konshin]

What if there will appear a 4.x tag? All canaries will become for 5.x?

[evocateur]

Well, I mean, once there is a reachable 4.x.x tag, then you can drop the premajor keyword argument (which defaults to prerelease when using --canary), and it will "just work".

[kirill-konshin]

Yep, of course. It just means that once we release 4.x we will have to update that too. Resolved.

But.

If it is a clean repo, with no tags, and I'd like to release something as canary, what to do?

[evocateur]

Release a 0.1.0 range? I'm unclear why it's impossible for you to release a normal version at all. --canary was originally intended as a "nightly" build, not a replacement for prerelease or other ways of signifying "in development" status.

[kirill-konshin]

Well, just imagine the case when developer wants to release something as nightly builds but lerna kinda forces the dev to have at least 0.0.1, which (in my opinion) should not be necessary for the given case.

[stephanepericat]

any updates ? i can still reproduce this issue with lerna 3.5 ... it seems that it expects the first publication to not be "canary", because it only fails if there are no prior tags. once a first "latest" version has been published, publishing to canary is no problem.

[fdlk]

It would be nice if the documentation at least warns about this, because now it was giving me headaches.

[tanguyantoine]

facing the same issue :-(

[jperasmus]

Facing the same issue. The reason we have this is that we had a completely separate repo previously that is currently published as v0.x - now we want to publish v1.0.0-alpha.x from this brand new monorepo. Is there a better way to release a prerelease than using the --canary flag?

[jperasmus]

Apologies, I just realized that I don't need the --canary flag to publish to a different dist-tag.

[phillipuniverse]

My situation is similar to the above, I have not made a release yet at all of my package. There are no tags, no semver releases on the NPM registry. I am trying to publish a canary build. I have package.json version set to 0.0.0 and was using this command with lerna 2.11:

yarn lerna publish --cd-version=patch --canary=snapshot --skip-git --npm-tag latest --exact

This successfully published 0.0.1-snapshot.$sha to the NPM registry. Here's the relevant output in the prompt:

lerna info version 2.11.0
lerna info canary enabled
lerna info current version 0.0.0
lerna info Checking for updated packages...
lerna info Comparing with initial commit.
lerna info Checking for prereleased packages...

Changes:
 - @scope/private-package: 0.0.0 => 0.0.1-snapshot.723ade45 (private)
 - @scope/package-1: 0.0.0 => 0.0.1-snapshot.723ade45
 - @scope/package-2: 0.0.0 => 0.0.1-snapshot.723ade45

After upgrading to 3.13.1 I updated this command to do the following instead:

yarn lerna publish --registry <private-registry-url> --canary --no-git-tag-version --no-push --force-publish='*' --dist-tag latest --preid snapshot patch

Which now attempts to set the version to null-snapshot.<something>+<sha>:

lerna notice cli v3.13.1
lerna info canary enabled
lerna WARN force-publish all packages
lerna info Assuming all packages changed

Found 2 packages to publish:
 - @scope/package-1 => null-snapshot.2625+04378626
 - @scope/package-2 => null-snapshot.2625+04378626

Lerna 2.x allowed me to release canary patches without having any tags in the repository, and thus without having done a single release before. Does Lerna 3.x not allow this?

[phillipuniverse]

Since a publish seems to be similar-ish to a lerna version (docs indicate it invokes it under the covers), it seems like there might be some divergence here. This lerna version part correctly ups the 0.0.0 version to 0.0.1-snapshot:

$ yarn lerna version --no-push --preid snapshot prepatch

lerna notice cli v3.13.1
lerna info current version 0.0.0
lerna info Assuming all packages changed

Changes:
 - @scope/private-package: 0.0.0 => 0.0.1-snapshot.0 (private)
 - @scope/package-1: 0.0.0 => 0.0.1-snapshot.0
 - @scope/package-2: 0.0.0 => 0.0.1-snapshot.0

I also tried creating a v0.0.0 tag - same result with null- version.

So I guess that's one workaround, use lerna version --no-push and then lerna publish afterwards?

Another workaround could be to create a first formal release, or at least a formal tag that is _greater than_ v0.0.0. Then everything started working as I would expect it to at version 0.0.2. If that's the case, then perhaps the right fix is to just document this as lerna publish will not work without an initial formal release to the registry?

[phillipuniverse]

Final comment, which I think would definitely categorize this as a bug and a regression in --canary builds.

I tried doing a publish _without_ --canary and the versions were correct:

yarn lerna publish --registry <registry> --no-git-tag-version --no-push --force-publish='*' --dist-tag latest --preid snapshot prepatch
yarn run v1.15.2
lerna notice cli v3.13.1
lerna info current version 0.0.0
lerna notice FYI git repository validation has been skipped, please ensure your version bumps are correct
lerna WARN force-publish all packages
lerna info Assuming all packages changed
lerna WARN version Skipping working tree validation, proceed at your own risk

Changes:
 - @scope/private-package: 0.0.0 => 0.0.1-snapshot.0 (private)
 - @scope/package-1: 0.0.0 => 0.0.1-snapshot.0
 - @scope/package-2: 0.0.0 => 0.0.1-snapshot.0

[kirill-konshin]

I have one more issue.

Last tag was 3.1.2: https://github.com/ringcentral/ringcentral-js/releases/tag/3.2.2
Last alpha tag 4.0.0-alpha.5 https://github.com/ringcentral/ringcentral-js/releases/tag/4.0.0-alpha.5

Travis called canary publish command like so https://github.com/ringcentral/ringcentral-js/blob/master/package.json#L14

How is that possible that I got THIS in my logs:

> lerna publish --tag-version-prefix="" --force-publish=* --canary "--yes"
lerna notice cli v3.4.3
lerna info ci enabled
lerna info canary enabled
lerna WARN force-publish all packages
lerna info Assuming all packages changed
Found 4 packages to publish:
 - @ringcentral/react => 3.1.2-alpha.30+7593d24
 - @ringcentral/redux => 3.1.2-alpha.30+7593d24
 - @ringcentral/sdk => 3.1.2-alpha.30+7593d24
 - @ringcentral/subscriptions => 3.1.2-alpha.30+7593d24

Note 3.1.2 and note alpha.30.

Lerna.json version https://github.com/ringcentral/ringcentral-js/blob/master/lerna.json#L14
Package.json version of each of published package is https://github.com/ringcentral/ringcentral-js/blob/master/sdk/package.json#L3

[kirill-konshin]

I guess I know why, Lerna was just too smart and picked up 3.1.2 from master branch whereas previous alphas were released from rollup branch. Yet still I don’t understand why canary release got alpha.30...

[kirill-konshin]

Also I don't understand how can canary pick up wrong tag like here:

dist-tags:
feature-npm: 19.4.1-feature-npm.4268.2  latest: 19.4.2 

Why 19.4.1??? I think there's too much magic involved here, I'd prefer to pass version explicitly. Either through main package.json or via lerna.json.

[starpit]

fwiw, we just switched from independent versioning to lockstep versioning. prior to this change, our --canary publishes had been working fine. with the lockstep change, our lerna publish --canary now fails with odd "null" versions, e.g.

@kui-shell/core => null-dev.3908.49+9ff3c79

[starpit]

Update: ok i see the underlying issue. with an independent versioning scheme, the lerna publish logic does not rely on git describe. whereas, with a fixed versioning scheme, it does. this means that, if your repo does not have an annotated tag that matches "v..*" for the branch you wish to publish, one that the git describe algorithm can find, ... then lerna publish fails with the "null" version bug.

our use case: our master branch version has no annotated tags, because we haven't yet published anything. i.e. master is truly the canary; we use version branches to track semvers. we want to prepublish a canary for our (not yet published) master, and this breaks lerna when run in a non-independent/fixed versioning scheme.

our hacky fix: push an annotated tag (it must be an annotated tag, not a lightweight tag) that satisfies lerna's tag-match criterion.

[kirill-konshin]

@evocateur is it technically possible to provide a version explicitly for the canary publish like so:

lerna publish 1.2.3 --canary --preid foo.X --dist-tag foo

Currently it results in an error:

lerna ERR! Error: Invalid tag name "^null-foo.1+ce84d175": Tags may not have any characters that encodeURIComponent encodes.

Where ^null is coming from?

This would solve my issue.

Also, it is almost impossible to reliably detect which version should be base for canary, so explicit setting would be nice.

[starpit]

i'm not a lerna dev, only a user. this part of lerna seems fairly buggy, unfortunately.

the null comes from lerna's attempt to infer the most recent "vx.y.z" tag for your current branch. it uses a git command (git describe) for this, and it is this git command that finds no matching tag. there is a bug in lerna publish, where it does not handle this situation gracefully.

this issues does not occur with independent versioning, as lerna's algorithm for inferring the versions there is different, and quite a bit more reliable. it is only with uniform versioning where the git describe algorithm gets involved, and to ... not such great effect.

as to why the situation occurs? it is frankly a bit of a mystery to me. there seems to be issues with publishing new versions in version branches, where lerna's heuristics fail: a subsequent lerna publish from master, after a semver publish in a verison branch... usually results in nulls, and i quite honestly have no idea how to resolve this reliably. but it means that every time i publish on a version branch.... master builds fail with nulls, and i have to scramble to figure out how to resolve this... each and every time.

in my opinion, when i'm doing a --canary prepatch publish in master (master is never doing semver releases, anyway)... git should not be consulted at all. the version in lerna.json should suffice to form the basis for the versions of the packages to be published.

[starpit]

FYI: the problem we have been having is in travis. travis does a shallow clone (--depth=50 seems to be its default). lerna uses git describe which, depending on your repo, may need more than 50 commits of history to infer the "last" version tag. adding a git fetch --depth=someBiggerNumber prior to the call to lerna publish seems to have helped us.

[kirill-konshin]

@evocateur may I ask you to pay a bit of attention to this topic.

I found another corner case when explicit version will be a must have for a canary release.

I have a Docker image with source code and in order to keep repo lean it does not have .git folder at all. All the needed files for publish are present. I don't see any valid reasons to force users to have git for sake of any automation. It's good if it's available, no big deal if not.

In order to overcome the limitation I have to do nasty stuff like this:

git init
git add .
git commit -m '1.0.0'
git tag -a '1.0.0' -m '1.0.0'
lerna publish --canary ...etc

[evocateur]

--canary has always depended on git to determine what changed and how to version it. If your Docker thingy has no use for git, I would take your explicit version and skip --canary altogether:

# --no-git-tag-version "turns off" all git operations for `lerna version`
npx lerna version ${MY_EXPLICIT_VERSION} --no-git-tag-version --exact --force-publish --yes

# "from-package" is the only bump argument for `lerna publish` that does not require git
npx lerna publish from-package --yes

It's a "global" release, but then I'm not sure how you'd expect to pick and choose without git around to tell you what's changed.

[kirill-konshin]

@evocateur thank you for prompt reply.

The thing is in my case lerna publish is in essence just lerna run publish with perk of setting the right version everywhere, no change log, no anything else. Just like what you described.

Looks like the expectation of publish command is that it's like a version command + publish. This impression comes from earlier Lerna releases.

Your recommendation at this moment is to prefer to separately call those commands, right?

I assume that in your example MY_EXPLICIT_VERSION can be anything like 1.0.0-canary.XXX?

[evocateur]

Looks like the expectation of publish command is that it's like a version command + publish.

Yep, this is the historical behavior of Lerna, inherited from Yarn. This conflation will be removed in the next major of Lerna. lerna version and lerna publish will then have the same relationship as npm version and npm publish, with the same ability to "chain" them (if desired) via lifecycle scripts.

Your recommendation at this moment is to prefer to separately call those commands, right?

In the example I posted, it's actually _required_ to call them separately, because there's no way to achieve the "make local changes without git and then publish" goal if you only call lerna publish. And, of course, in the next major, it'll also be required (because they'll be completely separate commands).

I assume that in your example MY_EXPLICIT_VERSION can be anything like 1.0.0-canary.XXX?

Correct!

[kirill-konshin]

Got it. Sounds good.

[JounQin]

Facing same issue on GitHub Actions, any news?

https://github.com/mdx-js/mdx/runs/410233372#step:8:50

[menosprezzi]

@JounQin &
Guys,

I'm using this command in my Github Actions CD in order to achieve the same behaviour of the --canary param:

lerna publish prerelease -y --force-publish=* --no-push --no-git-tag-version --dist-tag alpha

Since the --canary dont push any git tags, the --no-push and --no-git-tag-version accomplish that.
And the --dist-tag alpha will mark this version as an alpha release on NPM.

Hope it helps!

[JounQin]

@menosprezzi Great thanks, it works!
https://github.com/mdx-js/mdx/runs/419430015?check_suite_focus=true#step:10:468

[JounQin]

@menosprezzi It seems it always emits same sub version, did you meet this problem?
https://github.com/mdx-js/mdx/runs/422291101#step:10:69

[menosprezzi]

@JounQin ops. I didnt face this problem before. I see that your project use the same workflow for alpha and latest releases. This command should be used to alpha releases only cause it never will increment your root version.

We use another command to publish latest releases in npm:
lerna publish -y --force-publish=*

[menosprezzi]

@JounQin Sorry I didnt check the problem properly.

We are using the same version of Lerna. I think that the problem is that you're using the command with --pre-dist-tag ci --preid ci over --dist-tag ci. Maybe these params are skiping the check in the NPM for the previous published tag.

[JounQin]

That's wired, I tried --dist-tag ci, but it still throws error.

https://github.com/mdx-js/mdx/runs/424547300?check_suite_focus=true#step:10:69

[marikaner]

I was encountering the same issue with GitHub Actions. I was able to solve it by using the actions/checkout@v1 action instead of actions/checkout@v2 (version 2). Maybe this helps some of you folks.

I would assume that from lerna side it would be a valid improvement to just read the lerna.json version in case of fixed versions.

[ericclemmons]

Note, you'll also get NULL if you runlerna publish 0.0.0 ..., despite the version of actions/checkout.

[kautkata]

Same for us, shallow git clone via Jenkins and afterwards push to nexus

lerna publish --registry <some registry url> --legacy-auth <some creds | b64 encoded > --no-push --no-git-tag-version --no-commit-hooks --exact --yes --canary --preid head

lerna notice cli v3.20.2
lerna info ci enabled
lerna info canary enabled
lerna info Assuming all packages changed

Found 1 package to publish:
 - @foo-bar/common-ui => null-head.25+1d9ba43