Loopback-next: [CLI] Move away from using master to clone examples via `lb4 example`

[virkt25]

Description / Steps to reproduce / Feature proposal

When we have changes to an example repository that is dependent on changes in a published @loopback/* package, (feature / fix / anything breaking), between the time the PR is merged and a release is published to npm, any new examples cloned via lb4 example will break as it relies on code in master for dependencies.

This happened for 3 days in a row due to 3 different changes to our code base. Using master might be acceptable once we GA, but during alpha I think we need a better way of dealing with these changes.

See #1101, #1090, #1088

Option 1

• Publish example-* packages to npm and have the lb4 example script use the published package to clone the project instead of master OR even the latest tagged version on GitHub

Option 2

• Automate releases using Travis. If a PR is merged, Travis can automatically do a release.

Current Behavior

lb4 example getting-started will clone the project from master

Expected Behavior

lb4 example getting-started should clone the last released / stable version of the project so the project doesn't break till new versions of @loopback/* are released.

_See Reporting Issues for more tips on writing good issues_

Acceptance Criteria

• [x] Make all examples public npm packages again - remove publishConfig: {private:true} from package.json files (pull request: #1283)
• [x] Publish the first version of all examples to npmjs
• [ ] Rework lb4 example to fetch examples from npm using pacote library provided by npm, Inc.

[raymondfeng]

+1 to download the latest tagged/released tarball for examples.

I did some research and here are my findings:

1. https://api.github.com/repos/strongloop/loopback-next/releases returns an empty array. It hints that lerna publish does not create github releases.

2. https://api.github.com/repos/strongloop/loopback-next/tags gives us a list of tags but it cannot sort tags and we have to deal with pagination.

3. We can use github v4 apis to get latest tags but it requires a personal token from github.

curl -X POST -H 'Authorization: bearer <your-personal-token-without-scopes>' -H 'Content-Type: application/json' -H 'Accept: application/json' -d '{"query":"{\n  repository(owner: \"strongloop\", name: \"loopback-next\") {\n    refs(refPrefix: \"refs/tags/\", first: 50, orderBy: {field: TAG_COMMIT_DATE, direction: DESC}) {\n      edges {\n        node {\n          name\n        }\n      }\n    }\n  }\n}\n","variables":"{}","operationName":null}' 'https://api.github.com/graphql'

1. If we can list latest tags, we can use a url like https://api.github.com/repos/strongloop/loopback-next/tarball/@loopback/example[email protected] to download it.

[virkt25]

Great research @raymondfeng !!

I don't think we can use GitHub APIs via CLI because of the requirements of requiring a GitHub token :/

Thoughts on just publishing the example repos on npm and then the CLI can just use npm view to get the URL for the latest tarball for the package and download that. For example: https://registry.npmjs.org/@loopback/core/-/core-0.2.2.tgz

[raymondfeng]

I'm fine to use npm to manage versioning of examples.

[dhmlau]

ping @bajtos

[bajtos]

I think it's a great idea to switch from GitHub to npm. Beyond the problems described above, this change will also fix the problem when examples on master branch are using new APIs that have not been published to npm.

Downloading (and extracting!) tarballs from npm is super easy thanks to the official npm client library called pacote. It not only supports semver and downloads the correct version, but it also leverage npm's local cache and verifies checksum of downloaded tarballs.

Here is an example script to try out:

'use strict';
const pacote = require('pacote');

async function main() {
  await pacote.extract('@loopback/core@latest', './loopback4-core');
}

main().then(
  ok => console.log('DONE: ./loopback4-core'),
  err => console.error(err) && process.exit(1));

See their npm page for more details, including API docs: https://www.npmjs.com/package/pacote