Docusaurus: Github Actions deployment error

[pdimens]

🐛 Bug Report

(A clear and concise description of what the bug is)
Following the guidelines of GitHub Actions deployment here: https://v2.docusaurus.io/docs/deployment/#deploy
The actions fail to build and deploy the site with this error:

 Release to GitHub Pages5s
##[error]Process completed with exit code 1.
Run git config --global user.email "[email protected]"
npm ERR! cipm can only install packages with an existing package-lock.json or npm-shrinkwrap.json with lockfileVersion >= 1. Run an install with npm@5 or later to generate it, then try again.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/runner/.npm/_logs/2020-05-30T17_04_14_260Z-debug.log
##[error]Process completed with exit code 1.

To Reproduce

1. Added contents of documentation.yml from https://v2.docusaurus.io/docs/deployment/#deploy to file .github/workflows/documentation.yml in documentation branch of repository
2. Followed the instructions to generate/deploy ssh key and secret

Expected behavior

successfully build the site and deploy to branch gh-pages

Actual Behavior

The actions fail to build and deploy the site with this error:

 Release to GitHub Pages5s
##[error]Process completed with exit code 1.
Run git config --global user.email "[email protected]"
npm ERR! cipm can only install packages with an existing package-lock.json or npm-shrinkwrap.json with lockfileVersion >= 1. Run an install with npm@5 or later to generate it, then try again.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/runner/.npm/_logs/2020-05-30T17_04_14_260Z-debug.log
##[error]Process completed with exit code 1.

Your Environment

• Docusaurus version used: 2.0.0-alpha.56
• Environment name and version (e.g. Chrome 78.0.3904.108, Node.js 10.17.0): Node v14.2.0
• Operating system and version (desktop or mobile): Desktop Arch Linux

Reproducible Demo

https://github.com/pdimens/PopGen.jl/tree/documentation/.github/workflows

[lex111]

@artemkovalyov could you please look at it?

[anshulrgoyal]

I think the error is originating from missing package-lock.json since npm ci requires a lock file. And Check action is also not configured properly.
The following should fix check job

    steps:
      - uses: actions/checkout@v1
      - uses: actions/setup-node@v1
        with:
          node-version: '12.x'
      - name: Deploy
        run: |
            npm ci
            npm run build

[anshulrgoyal]

I will make a pr for check job and checking if a lock file is present before using npm ci and falling back to npm i

[pdimens]

Given that yarn was used and created yarn.lock rather than package-lock.json, is that the origin of the issue for the missing package-lock.json?

[anshulrgoyal]

npm ci is used in action so it will look for package-lock.json file i think so.

[anshulrgoyal]

@pdimens npm doesn't recognize yarn.lock file, I will try to change npm to yarn automatically in the action based on the lock file

[pjoshi751]

Hi.. I am facing the same issue and cannot find package-lock.json file when I install docusaurus via
npx @docusaurus/init@next init [name] [template] as given in the documentation.

[slorber]

Hey,

Using NPM ?

If you don't have a package-lock.json file, do you use npm locally?

If yes, have you run npm install locally?

Since npm5, all npm installs normally create a package-lock.json file.
Maybe you should upgrade to npm5 if you are using an older version (there are options to create a package-lock.json for older versions afaik)

Have you committed this lockfile? you should.

Using Yarn ?

Then you have a yarn.lock, and it would be better to use yarn in the CI to be sure to use the same versions in dev and CI (those in the lockfile)

So you should replace all NPM commands by yarn equivalent

yarn install --frozen-lockfile , according to https://stackoverflow.com/questions/58482655/what-is-the-closest-to-npm-ci-in-yarn

so when you see:

          run: |
            npm ci
            npm run build

You can replace by:

          run: |
            yarn install --frozen-lockfile
            yarn build

Don't forget to install yarn in the Github CI first: https://codyogden.blog/yarn-with-github-actions-ci-cd/

[pdimens]

Following @slorber suggestion, I modified the GitHub Actions workflow to:

name: documentation

on:
  pull_request:
    branches: [documentation]
  push:
    branches: [documentation]

jobs:
  checks:
    if: github.event_name != 'push'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - uses: actions/setup-node@v1
        with:
          node-version: '12.x'
      - run: npm install -g yarn
      - name: Deploy
        run: |
          yarn install --frozen-lockfile
          yarn build
  gh-release:
    if: github.event_name != 'pull_request'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - uses: actions/setup-node@v1
        with:
          node-version: '12.x'
      - name: Add key to allow access to repository
        env:
          SSH_AUTH_SOCK: /tmp/ssh_agent.sock
        run: |
          mkdir -p ~/.ssh
          ssh-keyscan github.com >> ~/.ssh/known_hosts
          echo "${{ secrets.GH_PAGES_DEPLOY }}" > ~/.ssh/id_rsa
          chmod 600 ~/.ssh/id_rsa
          cat <<EOT >> ~/.ssh/config
          Host github.com
          HostName github.com
          IdentityFile ~/.ssh/id_rsa
          EOT
      - name: Release to GitHub Pages
        env:
          USE_SSH: true
          GIT_USER: git
        run: |
          git config --global user.email "[email protected]"
          git config --global user.name "gh-actions"
          npm ci
          npx docusaurus deploy

and the result now seems to be an authentication error. I am under the impression that I followed the documentation about this process correctly, but now I'm unsure.

Cloning into 'PopGen.jl-gh-pages'...
Warning: Permanently added the RSA host key for IP address '140.82.112.3' to the list of known hosts.
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Error: Error: git clone failed
    at runDeploy (/home/runner/work/PopGen.jl/PopGen.jl/node_modules/@docusaurus/core/lib/commands/deploy.js:80:19)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
##[error]Process completed with exit code 1.

[slorber]

@pdimens

fatal: Could not read from remote repository.

That does not seem related to docusaurus, you should refer to Github Actions doc for that imho. Maybe you have a problem with your GH_PAGES_DEPLOY, but I don't know, we are not Github CI experts here :)

Deploying to Github Pages looks really complicated to me.
I'd recommend you to try using Netlify (or Vercel), this is way simpler, free, has much more features, better UX, PR deployment previews...

---

@yangshun @JoelMarcey current docs seems to recommend Github Pages (at least it's the first thing users see). What about recommending something else, that is also free, and does not require complex CI setup? Github pages was nice a few years ago, as it was the only free static hosting provider, but since then, the market has changed a lot...

[anshulrgoyal]

Following @slorber suggestion, I modified the GitHub Actions workflow to:

name: documentation

on:
  pull_request:
    branches: [documentation]
  push:
    branches: [documentation]

jobs:
  checks:
    if: github.event_name != 'push'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - uses: actions/setup-node@v1
        with:
          node-version: '12.x'
      - run: npm install -g yarn
      - name: Deploy
        run: |
          yarn install --frozen-lockfile
          yarn build
  gh-release:
    if: github.event_name != 'pull_request'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - uses: actions/setup-node@v1
        with:
          node-version: '12.x'
      - name: Add key to allow access to repository
        env:
          SSH_AUTH_SOCK: /tmp/ssh_agent.sock
        run: |
          mkdir -p ~/.ssh
          ssh-keyscan github.com >> ~/.ssh/known_hosts
          echo "${{ secrets.GH_PAGES_DEPLOY }}" > ~/.ssh/id_rsa
          chmod 600 ~/.ssh/id_rsa
          cat <<EOT >> ~/.ssh/config
          Host github.com
          HostName github.com
          IdentityFile ~/.ssh/id_rsa
          EOT
      - name: Release to GitHub Pages
        env:
          USE_SSH: true
          GIT_USER: git
        run: |
          git config --global user.email "[email protected]"
          git config --global user.name "gh-actions"
          npm ci
          npx docusaurus deploy

and the result now seems to be an authentication error. I am under the impression that I followed the documentation about this process correctly, but now I'm unsure.

Cloning into 'PopGen.jl-gh-pages'...
Warning: Permanently added the RSA host key for IP address '140.82.112.3' to the list of known hosts.
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Error: Error: git clone failed
    at runDeploy (/home/runner/work/PopGen.jl/PopGen.jl/node_modules/@docusaurus/core/lib/commands/deploy.js:80:19)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
##[error]Process completed with exit code 1.

It wouldn't work if u have passcode on ssh key

[pdimens]

@anshulrgoyal I don't know if I do (pretty new to this) and don't recall setting it up in such a manner. I may have to retry the instructions on the docusaurus site.

[anshulrgoyal]

@anshulrgoyal I don't know if I do (pretty new to this) and don't recall setting it up in such a manner. I may have to retry the instructions on the docusaurus site.

I have forked your repo and setted up GitHub action there using two different methods. U can try them. One of then doesn't require any config

[anshulrgoyal]

@pdimens this should work out of box https://github.com/anshulrgoyal/PopGen.jl/blob/documentation/.github/workflows/test.yml

[pdimens]

@anshulrgoyal thank you! I still seem to have deployment/authentication issues though, but it's probably something incorrectly configured on my part?

Deploy command invoked ...
HEAD
https://github.com/pdimens/PopGen.jl
ee7b7cb2e32a66be4d4a2c117264fd6163976472
Creating an optimized production build...
[info] [webpackbar] Compiling Client
[info] [webpackbar] Compiling Server
[success] [webpackbar] Client: Compiled successfully in 25.64s
[success] [webpackbar] Server: Compiled successfully in 29.16s

Success! Generated static files in build.

Cloning into 'PopGen.jl-gh-pages'...
Warning: Permanently added the RSA host key for IP address '140.82.113.3' to the list of known hosts.
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Error: Error: git clone failed
    at runDeploy (/home/runner/work/PopGen.jl/PopGen.jl/node_modules/@docusaurus/core/lib/commands/deploy.js:80:19)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
##[error]Process completed with exit code 1.

[pdimens]

@anshulrgoyal to follow up, I tried to see if it would make a difference to set up an SSH key for [email protected] rather than the email account associated with my github account, and that seemed to do the trick. I pushed a minor commit and GitHub Actions finally ran the job successfully. If what I did was the the correct method all along, I recommend adding the specificity in the docusaurus deploy docs of creating an SSH key for [email protected] .

[shinebayar-g]

This action works just fine without creating SSH key, using default GITHUB_TOKEN key.
I think docusaurus' yarn deploy script could achieve the same thing.

[stevenguh]

This an example github action that I am using to deploy to gh pages with peaceiris/actions-gh-pages. This action works fine without a new deploy ssh key.

name: deploy

on:
  pull_request:
    branches: [master]
  push:
    branches: [master]

jobs:
  checks:
    if: github.event_name != 'push'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - uses: actions/setup-node@v1
        with:
          node-version: '12.x'
      - name: Test Build
        run: |
          if [ -e yarn.lock ]; then
          yarn install --frozen-lockfile
          elif [ -e package-lock.json ]; then
          npm ci
          else
          npm i
          fi
          npm run build
  gh-release:
    if: github.event_name != 'pull_request'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - uses: actions/setup-node@v1
        with:
          node-version: '12.x'
      - name: Build
        run: |
          if [ -e yarn.lock ]; then
          yarn install --frozen-lockfile
          elif [ -e package-lock.json ]; then
          npm ci
          else
          npm i
          fi
          npm run build
      - name: Release to GitHub Pages
        uses: peaceiris/actions-gh-pages@v3
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          publish_dir: ./build