Aws-codebuild-docker-images: Ubuntu-Standard 4.0-21.01.08 introduces Segmentation Fault Error

[zhujik]

Latest Ubuntu Standard Release (2021-01-08) broke our build pipelines with the following error:

 [Container] 2021/01/11 07:44:41 Entering phase BUILD
 [Container] 2021/01/11 07:44:41 Running command git clone --quiet --bare --branch [REDACTED] https://git-codecommit.eu-central-1.amazonaws.com/v1/repos/[REDACTED];
 Segmentation fault
 [Container] 2021/01/11 07:44:42 Command did not exit successfully git clone --quiet --bare --branch [REDACTED] https://git-codecommit.eu-central-1.amazonaws.com/v1/repos/[REDACTED]; exit status 139
 [Container] 2021/01/11 07:44:42 Phase complete: BUILD State: FAILED
 [Container] 2021/01/11 07:44:42 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: git clone --quiet --bare --branch [REDACTED] https://git-codecommit.eu-central-1.amazonaws.com/v1/repos/[REDACTED];. Reason: exit status 139

Switching back to the prior version (aws/codebuild/standard:4.0-20.09.14) fixed this issue.

The issue is reproducable when calling this git command on multiple pipelines.

[xgnl]

Hi zhujik,

I've attempted to recreate the error in eu-central-1 on both standard:4.0-21.01.08 and standard:4.0-20.09.14 by running the following command: git clone --quiet --bare --branch 20.03.13 https://github.com/aws/aws-codebuild-docker-images.git. However, both builds ran successfully. If you can provide me with the build ARN of the segfaulted build, I can look deeper into the issue within our logs. If you feel comfortable sharing it with me here's my email: [email protected]

[ramakrishnakommuri]

I too have faced similar issue, by switching back to the prior version (aws/codebuild/standard:4.0-20.09.14) has fixed the issue. Thanks @zhujik

[zhujik]

@xgnl I sent you an email with two build ARNs. Thanks!

[xgnl]

Hi @zhujik and @ramakrishnakommuri,

We were able to isolate the issue to git version 2.30.0. If the core.bare git config is set to false and the --bare flag is used to clone the repo, it will cause a segfault. As a workaround, please add the following in your buildspec before cloning the repo: - git config --global --unset core.bare.

[zhujik]

@xgnl that is good news. Good job on finding that bug!

[subinataws]

@xgnl - are we going to change the default config of git or adding this to troubleshooting guide for CodeBuild? If it is the latter, we can resolve this issue.

[xgnl]

We will add it to the troubleshooting guide. I'm closing the issue.

[subinataws]

Bug report filed for this issue and we expect that to be corrected in a future release of git. https://lore.kernel.org/git/[email protected]/

[zhujik]

little update for anyone coming here: The proposed workaround now seems to throw an exit status 5 error, so just remove the workaround again.