jx gc previews fails on http gitlab

[dimmel82]

Summary

The jx gc previews command fails with error "error: no Git service resource found with URL 'https://mygitlab:9010' in namespace jx". The actual Gitlab url is http://mygitlab:9010

Steps to reproduce the behavior

1. Create an application with jx create spring -d web -d actuator command
2. Push changes on a PR-xx branch of the application
3. Execute command jx gc previews, or check the logs of the gc-previews pods.

Jx version

The output of jx version is:

NAME               VERSION
jx                 1.3.688
jenkins x platform 0.0.3125
Kubernetes cluster v1.10.0
kubectl            v1.13.1
helm client        v2.11.0+g2e55dbe
helm server        v2.12.0+gd325d2a
git                git version 1.8.3.1
Operating System   CentOS Linux release 7.5.1804 (Core)

Kubernetes cluster

What kind of Kubernetes cluster are you using & how did you create it?
Using existing minikube cluster

Operating system / Environment

CentOS Linux release 7.5.1804

Expected behavior

The jx gc previews command is successfully executed, and the gc-previews pods are completed

Actual behavior

The jx gc previews command fails and hundreds of gc-previews pods remain in error state

[dimmel82]

Note that other Gitlab operations are successfully completed by other jx commands, such as creation of repos during execution of jx create quickstart and jx create env

[iamonkara]

I don't think Jenkins-X folks are interested in supporting GitLab (or even bitbucket for that matter) as first class citizens like they do with GitHub. I mean look at the numbers of problems and their nature being reported, most problems involving GitLab and BitBucket is basic stuff that works right off the bat with GitHub

[dimmel82]

According to the Jenkins X roadmap the Gitlab integration is still in progress, so hopefully all the issues will be resolved some time soon

[jstrachan]

@iamonkara we absolutely do want to support GitLab + BitBucket as well as GitHub - its just GitHub was implemented first as we use it ourselves for Jenkins X itself so its easier to dogfood

[jstrachan]

@dimmel82 whats the output of jx get git server for you? Did you use http when adding your gitlab server? not sure why its getting confused between http and https

[dimmel82]

The output of jx get git server is:

Name   Kind   URL
GitHub github https://github.com
gitlab gitlab http://mygitlab:9010

The server was created with command jx create git server gitlab http://mygitlab:9010 -n gitlab

[dimmel82]

Updated the "Steps to reproduce the behavior" section because the error occurs only when at least one PR application exists.

[dimmel82]

Well, for some reason the automatically created environment root-demo-pr-feature1 is configured to use https.

kubectl describe env root-demo-pr-feature1 -n jx

Name: root-demo-pr-feature1
Namespace: jx
Labels:
Annotations:
API Version: jenkins.io/v1
Kind: Environment
Metadata:
Cluster Name:
Creation Timestamp: 2019-01-06T23:51:27Z
Generation: 1
Resource Version: 2189556
Self Link: /apis/jenkins.io/v1/namespaces/jx/environments/root-demo-pr-feature1
UID: fbb6e77f-120d-11e9-8bc3-005056946184
Spec:
Kind: Preview
Label: root/demo PR-feature1
Namespace: jx-root-demo-pr-feature1
Order: 999
Preview Git Info:
App Name: demo
Application URL: http://demo.jx-root-demo-pr-feature1.myminikube.nip.io
Name: feature1
URL: https://mygitlab:9010/root/demo/pull/feature1
User:
Promotion Strategy: Auto
Pull Request URL: https://mygitlab:9010/root/demo/pull/feature1
Source:
Kind: Git
Ref: HEAD
URL: https://mygitlab:9010/root/demo.git
Team Settings:
Status:
Events:

After editing the environment to replace https with http, a new error occurs:

Unable to convert PR feature1 to a number
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x12e2666]

goroutine 1 [running]:
github.com/jenkins-x/jx/pkg/gits.(GitlabProvider).GetPullRequest(0xc000c6a000, 0xc000a62c4a, 0x4, 0xc0009f7680, 0x0, 0x283bf04, 0xd, 0xc000c1f350)
/home/jenkins/go/src/github.com/jenkins-x/jx/pkg/gits/gitlab.go:287 +0xd6
github.com/jenkins-x/jx/pkg/jx/cmd.(GCPreviewsOptions).Run(0xc0009eb200, 0x0, 0x0)
/home/jenkins/go/src/github.com/jenkins-x/jx/pkg/jx/cmd/gc_previews.go:118 +0x471
github.com/jenkins-x/jx/pkg/jx/cmd.NewCmdGCPreviews.func1(0xc000578a00, 0x4622d30, 0x0, 0x0)
/home/jenkins/go/src/github.com/jenkins-x/jx/pkg/jx/cmd/gc_previews.go:62 +0x66
github.com/spf13/cobra.(Command).execute(0xc000578a00, 0x4622d30, 0x0, 0x0, 0xc000578a00, 0x4622d30)
/home/jenkins/go/pkg/mod/github.com/spf13/[email protected]/command.go:766 +0x2cc
github.com/spf13/cobra.(Command).ExecuteC(0xc00014d180, 0x0, 0x0, 0x0)
/home/jenkins/go/pkg/mod/github.com/spf13/[email protected]/command.go:852 +0x2fd
github.com/spf13/cobra.(*Command).Execute(0xc00014d180, 0xc0004a2d20, 0x2bddb20)
/home/jenkins/go/pkg/mod/github.com/spf13/[email protected]/command.go:800 +0x2b
github.com/jenkins-x/jx/cmd/jx/app.Run(0xc000877f88, 0xc0000ae058)
/home/jenkins/go/src/github.com/jenkins-x/jx/cmd/jx/app/jx.go:17 +0xdc
main.main()
/home/jenkins/go/src/github.com/jenkins-x/jx/cmd/jx/jx.go:10 +0x22

[erSitzt]

There is also the same problem with gc as in #3339 because the gc-previews pod is logging this as well:

Unable to convert PR serverport to a number
Can not get pull request serverport, skipping: GET https://srv-git.mydomain.com/api/v4/projects/323/merge_requests/0: 404 {message: 404 Not found}
Unable to convert PR fuckyou to a number
Can not get pull request fuckyou, skipping: GET https://srv-git.mydomain.com/api/v4/projects/351/merge_requests/0: 404 {message: 404 Not found}

[erSitzt]

I think i found the problem...

This is the output of jx get preview :

buehring@DESKTOP-V2ARMGS:~$ jx get preview
PULL REQUEST                                                             NAMESPACE                              APPLICATION
https://srv-git.mydomain.com/buehring/autodev-node/pull/serverport jx-buehring-autodev-node-pr-serverport http://autodev-node.jx-buehring-autodev-node-pr-serverport.172.16.200.81.nip.io
https://srv-git.mydomain.com/buehring/quickstart3/pull/justtesting jx-buehring-quickstart3-pr-justtesting http://quickstart3.jx-buehring-quickstart3-pr-justtesting.172.16.200.81.nip.io

But with GitLab they should look like this (and how its returned by the GitLab API) :

https://srv-git.mydomain.com/buehring/quickstart3/merge_requests/1
https://srv-git.mydomain.com/buehring/autodev-node/merge_requests/2

Thats why there are atoi-errors all over the place.

[dimmel82]

And regarding the "https" url in the preview env, it's because of fixed "https" value returned by PullRequestURL method https://github.com/jenkins-x/jx/blob/master/pkg/gits/git_url.go#L23,L25.

[erSitzt]

ok, my gitlab is actually https, so i didnt notice

seems to be very hardcoded in some places regarding github :(

[dimmel82]

@erSitzt you're right, it seems the go code for Gitlab needs some work to behave as for Github.
In Github the name of a PR can be same as the project's name, but in Gitlab the PRs are identified by numeric id.
Probably some additional command should get the PR number for the specific PR-xx branch.
The problem is that the number is only known after the PR is created, so the Jenkins pipeline has no way to know it.

[erSitzt]

But looking at the API of both GitHub and GitLab they are very similar

https://developer.github.com/v3/pulls/#list-pull-requests
https://docs.gitlab.com/ee/api/merge_requests.html

So why dont use the API to get the URLs für everything instead of building the url from hardcoded parts ? :)
GitLab supports access to a projects via api using the urlencoded projectname, so we can get everything we need from the repo-url.

https://srv-git.mydomain.com/api/v4/projects/buehring%2Fquickstart3/merge_requests

dont mind my last comment @jstrachan ... it was stupid after thinking about it :)

[jenkins-x-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://jenkins-x.io/community.
/lifecycle stale

[dimmel82]

/remove-lifecycle stale

[jenkins-x-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://jenkins-x.io/community.
/lifecycle stale

[jenkins-x-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://jenkins-x.io/community.
/lifecycle rotten

[jenkins-x-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://jenkins-x.io/community.
/close

[jenkins-x-bot]

@jenkins-x-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://jenkins-x.io/community.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the jenkins-x/lighthouse repository.