Pkgdown: deploy_to_branch() fails on Windows using multiple git bash setup options, could be a processx::run() problem

Created on 22 May 2020 · 18Comments · Source: r-lib/pkgdown

Hi,

Working with @lahuuki, we ran into a problem with pkgdown::deploy_to_branch() on Windows which I have now been able to reproduce on my Windows machine and well, that I'm very confused as to where the root of the issue is. It's very likely outside of pkgdown, but I'm reporting it here because pkgdown::deploy_to_branch() internally uses processx::run() to interact with git, while usethis relies on git2r, which does work on this Windows setup.

Given that I'll likely have to report this issue elsewhere, all the code lives at https://gist.github.com/lcolladotor/f5575143cf07f46cdb9cabc6ebe8d5c9.

Error description

Running pkgdown::deploy_to_branch() on a brand new package eventually ends up running pkgdown:::git("commit") but this fails. It fails because pkgdown internally uses processx::run("git") and for some reason unknown to me, under this scenario, git does not find the .gitconfig file that specifies the username and email for the computer user. Thus the error can be reproduced with:

## Create a package
usethis::create_package("~/Desktop/testgit")

## Set up git and say yes to the prompts
usethis::use_git()

## Create a file
writeLines("hola", "test.txt")

## Use git2r to add it (can also be done with the RStudio git panel)
git2r::add(path = "test.txt")

## Attempt to commit with the internal pkgdown code
pkgdown:::git("commit", "-m", "hola")
# Running git commit -m hola
#
# *** Please tell me who you are.
#
# Run
#
# git config --global user.email "[email protected]"
# git config --global user.name "Your Name"
#
# to set your account's default identity.
# Omit --global to set the identity only in this repository.
#
# fatal: unable to auto-detect email address (got 'fellg@DESKTOP-GB1AFPA.(none)')
# Error in processx::run("git", c(...), echo_cmd = echo_cmd, echo = echo,  :
#   System command 'git' failed, exit status: 128, stdout & stderr were printed
# Type .Last.error.trace to see where the error occured

Setup

R 4.0
Rtools40
RStudio version 1.2.5042 (latest stable release)
Install git bash https://git-scm.com/download/win. I have tested with all 3 options from

Capture

In all three options, RStudio and git2r::commit() did work, while processx::run("git") failed.

In the end, I have reverted back to my initial "Use git and optional Unix tools from the command prompt" option as recommended at https://github.com/rstudio/rstudio/issues/2224#issuecomment-368260312.

Major confusions

I still have no idea why processx::run("git") fails to find the user-contributed .gitconfig file under the different setups. Even the same git version (/cmd/git with processx https://gist.github.com/lcolladotor/f5575143cf07f46cdb9cabc6ebe8d5c9#file-debug_pkgdown-r-L89 and RStudio's terminal with git bash https://gist.github.com/lcolladotor/f5575143cf07f46cdb9cabc6ebe8d5c9#file-debug_pkgdown-r-L196) leads to different git config --list --show-origin outputs (processx one https://gist.github.com/lcolladotor/f5575143cf07f46cdb9cabc6ebe8d5c9#file-debug_pkgdown-r-L55-L85 vs https://gist.github.com/lcolladotor/f5575143cf07f46cdb9cabc6ebe8d5c9#file-debug_pkgdown-r-L199-L217) where processx::run("git") fails to find the C:/Users/fellg/.gitconfig file while RStudio's terminal with git bash does find it.

Even using the "Use git from git bash only" git bash installation route, I can make git commits with git2r and RStudio git's panel. Using the same git.exe path that RStudio uses (inspired from https://github.com/r-lib/processx/issues/161), I however end up with the same processx::run() error (the function used internally by pkgdown).

Moving forward

Maybe you have a better idea on how to find the root issue behind pkgdown/processx::run("git") on Windows (likely involves creating an issue at https://github.com/r-lib/processx/).
Maybe there are other Git for Windows setup options that you know of that we can try out (likely involves creating an issue at https://github.com/jennybc/happy-git-with-r/)
Maybe pkgdown could use git2r instead, like usethis. It seems to me that this would involve editing https://github.com/r-lib/pkgdown/blob/fcc120d996055519db89f37f0097af7ce25d6641/R/deploy-site.R#L222-L224 and the rest of the pkgdown:::git() calls in that script (I might be missing other ones). However, the git history for that file looks quite complicated and there's potentially other complications I'm unware of https://github.com/r-lib/pkgdown/commits/fcc120d996055519db89f37f0097af7ce25d6641?before=fcc120d996055519db89f37f0097af7ce25d6641+35&path%5B%5D=R&path%5B%5D=deploy-site.R. If you like this option, I could try to submit a PR for it.
One simpler option would be to rely on git2r for making the commits (or any other git step that requires having a username and email specified in the .gitconfig) and leave the rest unchanged (it would add git2r to the dependencies of pkgdown, which is maybe something you are trying to avoid doing).

Thank you again for your help r-lib!

Best,
Leo

Source

lcolladotor

Most helpful comment

I was hoping it would be related to some service degradation that https://githubstatus.com was reporting today, but it doesn't seem to be. I do think it's some sort of issue with the way that GitHub is setting up its git environment on its runners. As a workaround, I found that you can use the usethis package to manually establish your identity to git, as part of the recipe for the action. So, in my case, for example, my pkgdown.yaml workflow is taken pretty much directly from the example actions. In between the step where the package is built and pkgdown is called to push it to the gh-pages branch, add an extra step called, say, "Establish git credentials":

     - name: Install package
        run: R CMD INSTALL .

      - name: Establish git credentials
        run: |
          install.packages("usethis")
          usethis::use_git_config(user.name = "Kieran Healy", user.email = "[email protected]")
        shell: Rscript {0}  

      - name: Deploy package
        run: pkgdown::deploy_to_branch(new_process = FALSE)
        shell: Rscript {0}

As you can see we're just installing usethis if it's not already there and then calling use_git_config() to tell git our name and email. This seems to keep the runner happy during the Deploy package step, and the action completes as normal.

I think that in the ordinary course of things that user.name information, etc, would be auto-generated by the runner on start-up, but for whatever reason that's not happening.

kjhealy on 15 Jul 2020

👍5

All 18 comments

When attempting to rewrite the .yaml file generated with usethis::use_github_action("pkgdown") for Windows, I ran into the same problem in the final step with command pkgdown::deploy_to_branch(). While searching for a solution, I ran across this discussion where a workaround for the problem is presented. I am not sure how I can use it in my workflow, but I hope it can help to solve this problem in deploy_to_branch()?

ElsLommelen on 30 Jun 2020

I am having the same problem, initially the project worked just fine but then pkgdown started failing

brunocarlin on 15 Jul 2020

Very strange, I too have started to notice this exact error happening out of nowhere. I haven't changed the workflow at all: https://github.com/tyluRp/hierplane/actions?query=workflow%3Apkgdown

tyluRp on 15 Jul 2020

Tried to run on mac, windows and linux all systems fail on github actions

brunocarlin on 15 Jul 2020

Same here as of yesterday, with no changes to previously running workflows.

kjhealy on 15 Jul 2020

Similar issue with github action for me. Was fine yesterday but now failing as above.

tjtnew on 15 Jul 2020

@hadley have you seen this issue in other r-lib/tidyverse packages?

jayhesselberth on 15 Jul 2020

I think this might have been a temporary problem with GHA — I saw it yesterday but seems ok today

hadley on 15 Jul 2020

I am still getting the same error, running either locally or in gh-actions, my package is https://github.com/brunocarlin/shinyShare there were multiple successful builds up to yesterday

brunocarlin on 15 Jul 2020

Still failing here also.

kjhealy on 15 Jul 2020

It does seem like the issue isn't directly caused by some recent change in pkgdown ... replacing remotes::install_dev("pkgdown") with install.packages("pkgdown") in the Action recipe doesn't have any effect on what's happening.

kjhealy on 15 Jul 2020

👍1

     - name: Install package
        run: R CMD INSTALL .

      - name: Establish git credentials
        run: |
          install.packages("usethis")
          usethis::use_git_config(user.name = "Kieran Healy", user.email = "[email protected]")
        shell: Rscript {0}  

      - name: Deploy package
        run: pkgdown::deploy_to_branch(new_process = FALSE)
        shell: Rscript {0}

I think that in the ordinary course of things that user.name information, etc, would be auto-generated by the runner on start-up, but for whatever reason that's not happening.

kjhealy on 15 Jul 2020

👍5

Thanks @kjhealy, this is working on my end as well.

tyluRp on 15 Jul 2020

👍2

@kjhealy sounds like a good work around. We'll report back here if/when we figure out the underlying problem and fix it upstream.

hadley on 15 Jul 2020

if working collaboratively you may want to set something like user.name = "fix", user.email = "fix" to remove erroneous attribution of commit.

tjtnew on 16 Jul 2020

This was caused by a change in the upstream macOS image, which previously set the global user named and email,and now no longer seems to. We have update the image pkgdown workflow to set these for you now. You can update to the latest workflow with usethis::use_github_action("pkgdown").