Jx: Unexpected response from pipeline runner service: 400 Bad Request

[langleyd]

Summary

Running any quickstart directly after an apparently successful install fails at JX Promote

Message in staging repo PR:

unexpected response from pipeline runner service: 400 Bad Request,  ... visions, like this:
'git <command> [<revision>...] -- [<file>...]''

Pipeline runner log:
pielinerunner.txt

Looks like maybe shallow cloning is failing, as much as I can tell.

Steps to reproduce the behavior

1. Fresh install on gke
2. Create a quickstart

Expected behavior

Quicksart deploys ok.

Actual behavior

Fails at JX Promote, with errors from prow piplinerunner.

Jx version

The output of jx version is:

jx                 2.0.61
jenkins x platform 2.0.141
Kubernetes cluster v1.11.8-gke.6
kubectl            v1.11.9-dispatcher
helm client        Client: v2.13.1+g618447c
git                git version 2.11.0
Operating System   Debian GNU/Linux 9.8 (stretch)

Jenkins type

• [ ] Next Generation (Tekton + Prow)
• [ ] Classic Jenkins
• [x] Serverless Jenkins (JenkinsFileRunner + Prow)

Kubernetes cluster

Tried creating from Terraform and jx create cluster got the same issue from both.

Operating system / Environment

GKE cloud shell

[rawlingsj]

sounds like there's an issue with the pipelinerunner pod - could you check the logs and see if there's an error please?

[langleyd]

Logs are already attached above. As far as I can see the error is around git shallow clone failing( I'm new to all this so don't really have any context on what's going on yet):

I  Total 1138 (delta 6), reused 7 (delta 0), pack-reused 1119

E  shallow cloning pull request https://github.com/upswinggolf/environment-napered-upswingqa-staging.git of repository 1 to temp dir /tmp/git493273376

E  removing the temp directory /tmp/git493273376

E  WARNING:

E  retrying after error:shallow cloning pull request https://github.com/upswinggolf/environment-napered-upswingqa-staging.git of repository 1 to temp dir /tmp/git493273376

E  : failed to reset hard to  in directory /tmp/git493273376: git output: fatal: ambiguous argument 'origin/': unknown revision or path not in the working tree.

[langleyd]

I think this is a duplicate: https://github.com/jenkins-x/jx/issues/3800

[rawlingsj]

Yes we've seen this ourselves and will fix asap.
/priority critical-urgent
/area prow
/kind bug
/assign rawlingsj

[rawlingsj]

@jstrachan @pmuir we saw this earlier today before we manually switched our pipelinerunner image version to image: gcr.io/jenkinsxio/builder-maven:0.1.409.

[EamonKeane]

This looks partially resolved in 2.0.70 - the pipeline waits for you to manually merge it.

Is it possible to set up auto-merge for staging? This might be a sensible default for the demos as that's the way it used to be in jx 1.

[vfarcic]

It is indeed partially resolved in 2.0.70. Now builds to not fail, but wait indefinitelly for env. PR to merge. The issue is that PR to the env. repo does not initiate a build, so the merge of the env. PR (that one that was supposed to be automatic) never happens. Not only that the pipeline of the env. PR is not initiated but no other pipeline will run after the first build. In other words, only the first build works, and any afterward does not even start.

[dyusupov]

updated jx to 2.0.71 - did not help, same issue

[dyusupov]

manual pod edit and revert to gcr.io/jenkinsxio/builder-maven:0.1.409 from 0.1.423 helped.

[vfarcic]

Can we fix it to that specific tag as a workaround @rawlingsj ?

[EamonKeane]

@vfarcic everything working for me now after fresh install on 2.0.71. The default buildpack uses (gcr.io/jenkinsxio/builder-maven:0.1.433).

[vfarcic]

Works like a charm.

/close

[jenkins-x-bot]

@vfarcic: Closing this issue.

In response to this:

Works like a charm.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.