Jx: Bitbucket Cloud API deprecations

[wbrefvem]

Summary

Certain Bitbucket Cloud API endpoints will be removed on April 12, 2019 to comply with GDPR. Raising this issue to track any necessary changes to the Bitbucket Cloud provider, as well to go-bitbucket. More info here.

[amenic-hub]

Hi everyone, we are approaching the removal day. Is there any update with that regard?

For the sake of completeness, I post here the Atlassian Migration Guide reference
https://developer.atlassian.com/cloud/bitbucket/bbc-gdpr-api-migration-guide/

[lgg42]

How is this going on? I think this change has affected us great time! Since the "official" (wasn't April 12) change in the API our pipelines are broken during jx preview with error:

error: User.jenkins.io "" is invalid: metadata.name: Required value: name or generateName is required

We've tried a LOT of jx versions. If needed I can provide detailed info.

[lgg42]

Hi everyone, we are approaching the removal day. Is there any update with that regard?

For the sake of completeness, I post here the Atlassian Migration Guide reference
https://developer.atlassian.com/cloud/bitbucket/bbc-gdpr-api-migration-guide/

Have you been impacted by this change or things are running normal on your side?

[srtvasu]

We are also impacted by the same error.

error: User.jenkins.io "" is invalid: metadata.name: Required value: name or generateName is required

[srtvasu]

interesting observation.

jx preview on PR-* branch fails with the above error. But works on feature/* branch (git-flow) works.

For time being, I am trying to deploy into the preview for every commit on feature/* branch (not ideal though)

[lgg42]

Wow! Interesting! I'll try it when I can. Thanks for confirming this issue! We thought we were really the only ones using JenkinsX + BitBucket.

[lgg42]

interesting observation.

jx preview on PR-* branch fails with the above error. But works on feature/* branch (git-flow) works.

For time being, I am trying to deploy into the preview for every commit on feature/* branch (not ideal though)

Thanks A LOT for this interesting observation!! :raised_hands: :raised_hands: How did you even came up with this idea? :bulb:
It also works for me, though I also have to specify environment (--name) and namespace (--namespace) arguments. I think that with this hack we can unblock our pipelines until jx binary is updated for this issue!

[Veve2]

We also have multiple errors with Bitbucket cloud, with these "Unable to fetch user info" after the jx promote command, forcing us to abord manually a lot of build: (notice the empty space after "for", where the username should be)
{"level":"error","msg":"Unable to fetch user info for due to Status: 404 Not Found, Body: {\"type\": \"error\", \"error\": {\"message\": \"Resource not found\", \"detail\": \"There is no API hosted at this URL.\\n\\nFor information about our API's, please refer to the documentation at: https://developer.atlassian.com/bitbucket/api/2/reference/\"}

But far more problematic is the fact we can't install jx on a new cluster anymore, nor create new environments on our existing one*, jx install and jx create environment (for example) because of the 401 errors now returned by bitbucket API:

````
C:Users\myusername>jx create environment --git-provider-url=https://bitbucket.org --git-owner=repo-owner
? Name: test
? Label: Test
? Namespace: jx-test
? Domain: ourdomain.com
? Environment in separate cluster to Dev Environment: No

? Promotion Strategy: Auto
? We will now create a Git repository to store your test environment, ok? : Yes
Using Git provider bitbucket.org at https://bitbucket.org/
? Do you wish to use jenkinsx-bitbucketuser as the Git user name? Yes
X Sorry, your reply was invalid: Status: 401 Unauthorized, Body:
? Enter the new repository name: (environment-jx-test)
Revalidating the new repository name again, the error is then:
X Sorry, your reply was invalid: Status: 403 Forbidden, Body: Too many invalid password attempts. Log in at https://id.atlassian.com/ to restore access.
````

(We have to specify the git provider url because without it jx constently want to use github.com even if there isn't any mention of github.com in our .jx\gitAuth.yaml)

*: we've installed jx on this first cluster before the 29th April, date of the bitbucket API change.

At the moment we are sadly in a dead end as jx is unusable with Bitbucket cloud.

Here is the output of jx version, and my collegue on MacOS 10.14.5 has the same problem:
jx 2.0.298 jenkins x platform 0.0.3877 Kubernetes cluster v1.12.7 kubectl v1.14.0 helm client Client: v2.14.0+g05811b8 git git version 2.20.1.windows.1 Operating System Windows 10 Enterprise 1709 build 16299

Jenkins type

• [ ] Next Generation (Tekton + Prow)
• [X] Classic Jenkins
• [ ] Serverless Jenkins (JenkinsFileRunner + Prow)

Kubernetes cluster

AKS, created via portal.azure.com

[okgolove]

The error raises here https://github.com/jenkins-x/jx/blob/master/pkg/cmd/preview/preview.go#L282-L285

There is no email field in a response from Bitbucket. As far as I understand it has to be gotten using another function https://github.com/jenkins-x/jx/blob/ae0c16ca03873c53f8924d0ab1c68eab0574bde9/pkg/gits/bitbucket_cloud.go#L453

https://github.com/jenkins-x/jx/blob/master/pkg/cmd/preview/preview.go#L286

But it can't be validated because of 282-285 lines.

@pmuir @ccojocar @jstrachan
As far as I can see this is the really critical issue. For bitbucket users preview functional doesn't work at all.

UPD. it seems Login field has disappeared from bitbucket api response

[Veve2]

Almost all functions don't work anymore since bitbucket has changed the way to authenticate, using a user_id instead of the username now. And since the new API has the same version as the previous one, we're out of luck.

Extract from the deprecation notice, effective since the 29th of April:

Removal of usernames from user-referencing APIs

We are deprecating the use of username from user objects through the API. This affects URLs containing a username in the path on /users endpoints. As of 29 April 2019 usernames will no longer be supported in API URLs that reference users, and account_id or uuid should be used to identify an account instead

(for example, /2.0/users/557058:c0b72ad0-1cb5-4018-9cdc-0cde8492c443 vs /2.0/users/evzijst)

We are excluding repo URLs from this deprecation to avoid breaking repo references. As part of giving users better control of their data, we are going to separate references to user-spaces from content-spaces, which will preserve existing repo URLs. For more information on this, see Change notice: Updates to Bitbucket Cloud repository URL paths.

Replace username fields with Atlassian Account ID (account_id)

The Bitbucket's user object schema will lose its username field after 29 April 2019. This will affect clients that use username to identify accounts in API request objects.

For instance, instead of:
Copy

$ curl https://api.bitbucket.org/2.0/repositories/foo/bar/forks \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "my_fork",
    "owner": {
      "username": "evzijst"
    }
}'

do this instead:
Copy

$ curl https://api.bitbucket.org/2.0/repositories/foo/bar/forks \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "my_fork",
    "owner": {
      "account_id": "557058:c0b72ad0-1cb5-4018-9cdc-0cde8492c443"
    }
}'

@wbrefvem I've noticed you've made the bitbucketcloud go library, do you think an update of the functions called in jx\pkg\gits\bitbucket_cloud.go could be possible? Especially the auth one?

[srtvasu]

interesting observation.
jx preview on PR-* branch fails with the above error. But works on feature/* branch (git-flow) works.
For time being, I am trying to deploy into the preview for every commit on feature/* branch (not ideal though)

Thanks A LOT for this interesting observation!! 🙌 🙌 How did you even came up with this idea? 💡
It also works for me, though I also have to specify environment (--name) and namespace (--namespace) arguments. I think that with this hack we can unblock our pipelines until jx binary is updated for this issue!

Actually, it was an accident. We were in need of rewiring the preview mode for git-flow pipeline. I tried to run by modifying the branchpattern to include "feature/*" pattern. That's how we ended-up observing the results.

By the way, though this works better than the PR-*, it still has issues.

1. Jenkins X bot fails while committing the comments back to the branch
2. preview env is not getting recycled when I close the source "feature/*" branch.(this is important and I am still investigating)
3. "jx get preview --current" doesn't return the application URL

Long term solution is critical. Hoping it comes out soon.

[lgg42]

interesting observation.
jx preview on PR-* branch fails with the above error. But works on feature/* branch (git-flow) works.
For time being, I am trying to deploy into the preview for every commit on feature/* branch (not ideal though)

Thanks A LOT for this interesting observation!! raised_hands raised_hands How did you even came up with this idea? bulb
It also works for me, though I also have to specify environment (--name) and namespace (--namespace) arguments. I think that with this hack we can unblock our pipelines until jx binary is updated for this issue!

Actually, it was an accident. We were in need of rewiring the preview mode for git-flow pipeline. I tried to run by modifying the branchpattern to include "feature/*" pattern. That's how we ended-up observing the results.

By the way, though this works better than the PR-*, it still has issues.

1. Jenkins X bot fails while committing the comments back to the branch
2. preview env is not getting recycled when I close the source "feature/*" branch.(this is important and I am still investigating)
3. "jx get preview --current" doesn't return the application URL

Long term solution is critical. Hoping it comes out soon.

1. Are you talking about the previews or in general? we have to create our previews with --no-comment=true it has not worked for us yet, we think because of this: https://github.com/jenkins-x/jx/issues/3076
2. are your garbage collectors for previews working ok? like getting a completed after running?

[srtvasu]

interesting observation.
jx preview on PR-* branch fails with the above error. But works on feature/* branch (git-flow) works.
For time being, I am trying to deploy into the preview for every commit on feature/* branch (not ideal though)

Thanks A LOT for this interesting observation!! raised_hands raised_hands How did you even came up with this idea? bulb
It also works for me, though I also have to specify environment (--name) and namespace (--namespace) arguments. I think that with this hack we can unblock our pipelines until jx binary is updated for this issue!

Actually, it was an accident. We were in need of rewiring the preview mode for git-flow pipeline. I tried to run by modifying the branchpattern to include "feature/" pattern. That's how we ended-up observing the results.
By the way, though this works better than the PR-, it still has issues.

1. Jenkins X bot fails while committing the comments back to the branch
2. preview env is not getting recycled when I close the source "feature/*" branch.(this is important and I am still investigating)
3. "jx get preview --current" doesn't return the application URL

Long term solution is critical. Hoping it comes out soon.

1. Are you talking about the previews or in general? we have to create our previews with --no-comment=true it has not worked for us yet, we think because of this: #3076
2. are your garbage collectors for previews working ok? like getting a completed after running?

I was talking about only about preview mode. Yes. "--no-comment" option works.

Regarding GC for preview, we are not sure yet how gc triggers the resources to be recycled (env + namespace + deployment + svc etc). We are trying with default name and namespace, which is not getting recycled. We are thinking of changing the default name and namespace to see whether wit works for GC.

[enkicoma]

Hi guys,
Jx server

jx                2.0.206
jenkins x platform 2.0.672
Kubernetes cluster v1.12.6-eks-d69f1b
kubectl            v1.14.2
helm client        Client: v2.14.0+g05811b8
git                git version 2.21.0
Operating System   Mac OS X 10.14.5 build 18F
all the jobs fails...

Tried
jx promote -b --env develop --timeout 1h --version 1.0.115

got this error

ERROR: Unable to fetch user info for  due to Status: 404 Not Found, Body: {"type": "error", "error": {"message": "Resource not found", "detail": "There is no API hosted at this URL.\n\nFor information about our API's, please refer to the documentation at: https://developer.atlassian.com/bitbucket/api/2/reference/"}}
Created Pull Request: https://bitbucket.org/•••z•/environment-jx-develop/pull-requests/456
ERROR: Unable to fetch user info for  due to Status: 404 Not Found, Body: {"type": "error", "error": {"message": "Resource not found", "detail": "There is no API hosted at this URL.\n\nFor information about our API's, please refer to the documentation at: https://developer.atlassian.com/bitbucket/api/2/reference/"}} 

tried to run promote with --no-comment=true but is not accepting this flag...

Any idea how to fix these failures?

[gcameo]

Please this has left us dead in the water while in the middle of a client release? Can someone from bitbucket tell us what the work around is?

[Veve2]

There isn't any workaround, the solution here is to update the bitbucket cloud API library used by Jenkins X (https://github.com/wbrefvem/go-bitbucket), then JX itself, everywhere a username is used.

Or to use github instead of bitbucket, but it's not really a solution for many of us I imagine.

[srtvasu]

There isn't any workaround, the solution here is to update the bitbucket cloud API library used by Jenkins X (https://github.com/wbrefvem/go-bitbucket), then JX itself, everywhere a username is used.

Or to use github instead of bitbucket, but it's not really a solution for many of us I imagine.

+1

I don't see JX + Bitbucket-Cloud as usable combination till this is addressed.

[gcameo]

Well, I have been a paid bitbucket user for nearly 3 years... I guess its
time to move to Github then.
Hopefully, someone in the bitbucket team is listening. You can't just
upgrade a major API and break such a critical integration without even
caring...

On Fri, Jun 28, 2019 at 8:09 PM Vasu.dev@n [e-mail] wrote:

There isn't any workaround, the solution here is to update the bitbucket
cloud API library used by Jenkins X (
https://github.com/wbrefvem/go-bitbucket), then JX itself, everywhere a
username is used.

Or to use github instead of bitbucket, but it's not really a solution for
many of us I imagine.

+1

I don't see JX + Bitbucket-Cloud as usable combination till this is
addressed.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/jenkins-x/jx/issues/2149?email_source=notifications&email_token=AADN7N37QYDJLAHCZ64O52LP4ZOXVA5CNFSM4GBFVYE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY25YZI#issuecomment-506846309,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADN7N4X52QSYVUPYZGAPN3P4ZOXVANCNFSM4GBFVYEQ
.

[ozdemirkan]

I am having the same problem too. Most of the Jenkins X functionalities are broken with BitBucket cloud. Is there a plan to fix this problem any time soon? Having information about the fix plan will help us to choose the way forward.

error: finding existing PRs using filter on repo xxx/environment-xxx-staging: listing open pull requests on xxx/environment-xxx-staging: Status: 404 Not Found, Body: {"type": "error", "error": {"message": "Resource not found", "detail": "There is no API hosted at this URL.\n\nFor information about our API's, please refer to the documentation at: https://developer.atlassian.com/bitbucket/api/2/reference/"}}

[Matias-sys]

I am having the same issue, is there a solution or do we need to move all our repositories to github?

[anonimitoraf]

Has anyone tried https://github.com/jenkins-x/jx/issues/2149#issuecomment-506802366? I don't know in detail how our jenkins environment is setup but I'll post how I go with trying it out when I get to doing it.

[thienlh]

Any update on this issue?

[baskar030]

For me only jx promote alone throws 404 from bitbucket, so using the charts folder of microservice I directly used helm install / upgrade to promote the builds to kubernetes.

[infoatdfx]

Also having the same issue. A solution should come from the jx-team. This is a business critical issue for a lot fo people!

[lgg42]

Guys/Girls, I'm sure you are aware of this but I just want to mention it. Bitbucket is a principal git provider and many many Enterprise companies use it for it's awesome Jira/Confluence integration. And also many other non-enterprise too.
JenkinsX is a great and promising cloud native CICD solution and a lot of these companies craving cloud migrations might be looking at JenkinsX as a possible candidate. But the moment they find out (the very hard way, after many troubleshooting) that BitBucket integration is broken they are surely gonna ditch JenkinsX (take into account that the learning curve is not necessarily an easy one)
I think, that the more this issue is kept unresolved, JenkinsX might be loosing potentials users that even might become contributors afterwards.
Taking that into account, why isn't this issue getting some love?

[lgg42]

Hey all, I've noticed our gcpreview jobs are not doing their work
when running jx gc previews --batch-mode we get a :

WARNING: Unable to convert PR feature/branch-name to a number
WARNING: Can not get pull request feature/branch-name, skipping: Status: 401 Unauthorized, Body:

First time I see it, are you the ones using Bitbucket also experiencing this issue?

[valiksb]

@wbrefvem is there any way we can help. I dont know the go language, but happy to spend some hours learn it and see if we can fix this issue.

In the meantime others that are having the issue and resolved it with helm install/upgrade, could you guys provide the syntax used

[lgg42]

@wbrefvem is there any way we can help. I dont know the go language, but happy to spend some hours learn it and see if we can fix this issue.

In the meantime others that are having the issue and resolved it with helm install/upgrade, could you guys provide the syntax used

where have you read about this issue being resolved with helm install/upgrade?

[thienlh]

So jx is abandoning BitBucket now...

[gcameo]

Yesterday we started on journey to migrate to GitHub... We will never be
looking back... Sometimes it's surprising what kills businesses

On Thu, 26 Sep 2019, 11:06 Lê Hùng Thiện, [e-mail] wrote:

So jx is abandoning BitBucket now...

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/jenkins-x/jx/issues/2149?email_source=notifications&email_token=AADN7N2NCCPZXEYV6NFMDI3QLSCSNA5CNFSM4GBFVYE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7VBN2Q#issuecomment-535434986,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADN7N4OA2KS2WP3LSXIYYDQLSCSNANCNFSM4GBFVYEQ
.

[lgg42]

So jx is abandoning BitBucket now...

I wonder if they'll make an official announce or state it in the docs.

[thienlh]

So jx is abandoning BitBucket now...

I wonder if they'll make an official announce or state it in the docs.

That would be nicer tbh.

[jstrachan]

@thienlh here's docs on how to use Jenkins X with jx boot and different git providers: https://jenkins-x.io/docs/getting-started/setup/boot/#git

we've not had chance to test against Bitbucket Cloud yet though; but all of our tests are passing for github, github enterprise, bitbucket server & gitlab. I hope we can get Bitbucket Cloud supported too

[thienlh]

@thienlh here's docs on how to use Jenkins X with jx boot and different git providers: https://jenkins-x.io/docs/getting-started/setup/boot/#git

we've not had chance to test against Bitbucket Cloud yet though; but all of our tests are passing for github, github enterprise, bitbucket server & gitlab. I hope we can get Bitbucket Cloud supported too

But we don't use jx boot yet, we just want our current pipelines to run properly with BitBucket Cloud.

[yaariyuval]

@jstrachan this issue exists because clearly Bitbucket Cloud isn't supported and doesn't work. We all followed the documentation just fine. Suggesting otherwise and blaming it on your users is really not going to get anyone anywhere.
If you look closely at the issues on GitHub, GitLab isn't supported by jx, either, so it sounds like the problem is the tests not actually testing any basic functionality.

[jstrachan]

@yaariyuval I've never blamed any of our users for anything - I've no idea where you got that from.

To be clear github, github enterprise, bitbucket server & gitlab are all passing our BDD tests with jx boot (spinning up a cluster with all the environment repos on the git provider of choice, creating a quickstart, promoting to staging + production, creating a PR on the quickstart and validating a Preview is created). So we know Jenkins X works with boot + lighthouse + Jenkins X pipelines just fine.

Docs here: https://jenkins-x.io/docs/getting-started/setup/boot/#git

We've yet to do the same level of testing on bitbucket cloud - am sure there are things we'll need to fix to get it work. As usual in any OSS project - help figuring out how to get boot + lighthouse working with BitBucket Cloud would be greatly appreciated

[patrickwall57]

For me only jx promote alone throws 404 from bitbucket, so using the charts folder of microservice I directly used helm install / upgrade to promote the builds to kubernetes.

Were you getting a 404 that the api is not there or were you getting something like {"type": "error", "error": {"message": "Repository xxx/environment-xxx-staging not found"}}

[bakuppus]

Any solution for jx + bitbucket cloud ?

am getting error while setup multi cluster

kubectl logs environment-controller-5fd9b4c95f-92vqj
using require GitHub headers: true
WARNING: No $GIT_KIND defined or --git-kind supplied to assuming GitHub.com environment git repository
using environment source directory https://api.bitbucket.org/2.0/repositories/balafiverr/environment-development-staging and external webhook URL: https://api.bitbucket.org/2.0/repositories/balafiverr/environment-development-staging/hooks
verifying that the webhook is registered for the git repository https://api.bitbucket.org/2.0/repositories/balafiverr/environment-development-staging
error: failed to create git provider for git URL https://api.bitbucket.org/2.0/repositories/balafiverr/environment-development-staging: no Git service resource found with URL 'https://api.bitbucket.org' in namespace default

[jstrachan]

I just answered this on slack https://kubernetes.slack.com/archives/C9MBGQJRH/p1575964320105700?thread_ts=1575924556.100500&cid=C9MBGQJRH - environment controller is github only but lighthouse works great with bitbucket cloud

[jenkins-x-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://jenkins-x.io/community.
/lifecycle stale

[lgg42]

/remove-lifecycle stale

[jenkins-x-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://jenkins-x.io/community.
/lifecycle stale

[jenkins-x-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://jenkins-x.io/community.
/lifecycle rotten

[jenkins-x-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://jenkins-x.io/community.
/close

[jenkins-x-bot]

@jenkins-x-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://jenkins-x.io/community.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the jenkins-x/lighthouse repository.

[alexter42]

/remove-lifecycle stale.

I am getting
ERROR: Unable to fetch user info for * due to Status: 404 Not Found, Body: {"type": "error", "error": {"message": "*"}}
ERROR: Unable to fetch user info for * due to Status: 404 Not Found, Body: {"type": "error", "error": {"message": "*"}}
Pull Request https://bitbucket.org/project/environment-cluster-staging/pull-requests/2 is merged at sha 154bb6733cc3
Pull Request merged but we are not waiting for the update pipeline to complete!
WARNING: Could not find the service URL in namespace -staging for names http test2, * -http * test2, * -staging-http * test2

[lgg42]

/remove-lifecycle stale.

I am getting
ERROR: Unable to fetch user info for * due to Status: 404 Not Found, Body: {"type": "error", "error": {"message": "_*"}} ERROR: Unable to fetch user info for * due to Status: 404 Not Found, Body: {"type": "error", "error": {"message": "_"}}
Pull Request https://bitbucket.org/project/environment-cluster-staging/pull-requests/2 is merged at sha 154bb6733cc3
Pull Request merged but we are not waiting for the update pipeline to complete!
WARNING: Could not find the service URL in namespace -staging for names http test2, * -http * test2, * -staging-http ** test2

This issue is opened since 2 years already and no solution was ever given. I suggest you try the latest version of JX and use lighthouse.