Renovate: Bitbucket Support

[benderTheCrime]

Does renovate support Bitbucket at all? If not is there any plan to add support?

[rarkins]

Bitbucket is not currently supported, and I don't have any plans currently. However, the code is structured in such a way that adding new providers should be fairly easy from now on.

We added GitLab support a month or two ago. An example of the API required can be found in the GitLab API source: https://github.com/singapore/renovate/blob/master/lib/api/gitlab.js

If you're interested in helping, or for anyone reading this in future who is, you can see how we worked through GitLab support here: https://github.com/singapore/renovate/issues/65
Essentially, an interested user checked the GitLab API for compatibility and after that I wrote the code accordingly. I don't mind doing the same for Bitbucket if there's an interested user or two who I know will be around to test it, because I'm not an active user of Bitbucket myself.

[benderTheCrime]

This looks like it might be somewhat of a feat of strength. For one, it does not look like Bitbucket has auth token support.

I will do some research here on how this might be done, but also, if I'm the only one it may not be something worth asking you to build

[rarkins]

app passwords sounds like one way: https://confluence.atlassian.com/bitbucket/app-passwords-828781300.html

[jribeiro]

@rarkins have you had a chance to look into the requirements? I'd be keen to help out if needed.

[rarkins]

@jribeiro actually I will be looking into this again in the next week or two, I'll update here

[rarkins]

If you can examine how BitBucket's tokens/password support works and report back though, it would be a helpful start. Ideally we want something like the "personal access tokens" we use in GitHub, GitLab and VSTS

[jribeiro]

Hi @rarkins,

I've been looking at Bitbucket app passwords and they seem to work. The idea is that you create an app password and you associate specific permissions with it. In my test, adding write PR permissions also added read PR and read/write repository.

The above gives you back a token which you can use in combination with your bitbucket username to generate the Authentication: Basic <token> header.

The only pitfall with this is that in order to use API v2 - which I use in these examples - you need to have 2-factor authentication in place.

I've also compiled a list of endpoints similar to the list created for Gitlab:
BaseURL: https://api.bitbucket.org/2.0/

Please note that the repo name, in bitbucket, seems to be a combination of the username and the repo slug - ${owner}/${repoName}

initRepo

• GET repos/${repoName}
  
  
  
  • res.body.owner.login
  
  
  • res.body.default_branch
  
  
• GET /repositories/${owner}/${repoName}
  
  
  
  • res.body.owner.username
  
  
  • res.body.mainbranch.name
  
  

getBranchCommit

• GET repos/${config.repoName}/git/refs/heads/${branchName}
  
  
  
  • res.body.object.sha
  
  
• GET /repositories/${owner}/${config.repoName}/refs/branches/${branchName}
  
  
  
  • res.body.target.hash
  
  

getCommitTree

• GET repos/${config.repoName}/git/commits/${commit}
  
  
  
  • res.body.tree.sha
    
    Don't think this is actually supported atm. See: https://bitbucket.org/site/master/issues/9279/changed-files-between-commits-bb-10390
    
    When you request /repositories/${owner}/${repoName}/commits/${commitId} there's a few links including the diff which might be helpful (?)
  
  

getRateLimit

• GET rate_limit
  
  
  
  • res.body
    
    No end-point. 5k/hour for file req; 1k/hour for listing repos, commits, etc
    
    https://confluence.atlassian.com/bitbucket/rate-limits-668173227.html
  
  

findFilePaths

• GET search/code?q=repo:${config.repoName}+filename:${fileName}
  No known equivalent in bitbucket. It's possible to retrieve a list of all repository root files:
• GET /repositories/${owner}/${repoName}/src
  And then request sub-folders:
• GET /repositories/${owner}/${repoName}/src/${branchName}/${subFolder}

branchExists

• GET repos/${config.repoName}/git/refs/heads/${branchName}
• GET /repositories/${owner}/${repoName}/refs/branches/${branchName}

createBranch

• POST repos/${config.repoName}/git/refs
  Not supported in bitbucket cloud. You can create a branch when making a commit:
• POST /repositories/${owner}/${repoName}/src
  message
  branch
  https://developer.atlassian.com/bitbucket/api/2/reference/resource/repositories/%7Busername%7D/%7Brepo_slug%7D/src#post

deleteBranch

• DELETE repos/${config.repoName}/git/refs/heads/${branchName}
  Not supported in bitbucket

updateBranch

• PATCH repos/${config.repoName}/git/refs/heads/${branchName}
  No known equivalent, but the create branch end-point may work

getBranchPr

• GET repos/${config.repoName}/pulls?+state=open&base=${config.defaultBranch}&head=${config.owner}:${branchName}
  No known equivalent. It's possible to retrieve a list of all pull requests, and filter on that
• GET /repositories/${owner}/${repoName}/pullrequests/activity

addAssignee

• POST repos/${config.repoName}/issues/${issueNo}/assignees

Not sure but this may help: https://developer.atlassian.com/bitbucket/api/2/reference/resource/repositories/%7Busername%7D/%7Brepo_slug%7D/issues#post

addLabels

• POST repos/${config.repoName}/issues/${issueNo}/labels

Not supported from what I can see

findPr

• GET repos/${config.repoName}/pulls?head=${config.owner}:${branchName}

No known equivalent. It's possible to retrieve a list of all merge requests and filter on that https://developer.atlassian.com/bitbucket/api/2/reference/resource/repositories/%7Busername%7D/%7Brepo_slug%7D/pullrequests

checkForClosedPr

• GET repos/${config.repoName}/pulls?state=closed&head=${config.owner}:${branchName}

Same as above. You can pass state filter which can be one of MERGED / SUPERSEDED / OPEN / DECLINED
https://developer.atlassian.com/bitbucket/api/2/reference/resource/repositories/%7Busername%7D/%7Brepo_slug%7D/pullrequests

createPr

• POST repos/${config.repoName}/pulls
• POST /repositories/${owner}/${repoName}/pullrequests
  https://developer.atlassian.com/bitbucket/api/2/reference/resource/repositories/%7Busername%7D/%7Brepo_slug%7D/pullrequests#post

getPr

• PATCH repos/${config.repoName}/pulls/${prNo}
• PUT /repositories/${owner}/{repoName}/pullrequests/{pull_request_id}
  https://developer.atlassian.com/bitbucket/api/2/reference/resource/repositories/%7Busername%7D/%7Brepo_slug%7D/pullrequests/%7Bpull_request_id%7D#put

getFile/getFileContent/getFileJson

• GET repos/${config.repoName}/contents/${filePath}?ref=${branchName}
• GET /repositories/${owner}/{repoName}/downloads/{filename}

writeFile

• PUT repos/${config.repoName}/contents/${filePath}
• POST /repositories/${owner}/${repoName}/src

commitFile -> createBlob -> createTree -> createCommit
No known equivalent end-points for these functions. However, you may be able to create the required commit and file for that commit, using the commit end-point

Hope it helps.

[rarkins]

@jribeiro that's definitely really helpful and it looks very promising. Hopefully enabling 2FA is not a problem if people wish to utilise this with APIv2.

FYI I have moved the GitHub implementation away from using API find/search endpoints and instead doing as you suggested - downloading a complete list the first time it's needed and then caching it. This applies to the file list, PR list, and branch list. The app is generally limited by network/API time so reducing requests is really helpful.

I'd like to note here one more thing to look up: is BitBucket happy with forward slashes / in repository and branch names in the URL, or do they need them escaped with %2F. GitHub is fine with them but GitLab recently moved to escaping and it's been painful.

@iamstarkov has offered to fork and start a WIP soon, I will then fill out a kind of skeleton of functions necessary and any tips/comments for implementation. Combined with your above notes on the endpoints, we can hopefully get this done soon!

[rarkins]

We should also note here any peculiarities of BitBucket compared to others. It seems they follow the same naming conventions as GitHub (e.g. Pull Requests). I remember seeing previously that they only support markdown bodies for PRs though (not HTML), which is like VSTS.

[iamstarkov]

@jribeiro this is hell of a research

[iamstarkov]

@iamstarkov here, i will look into this starting from tomorrow

[iamstarkov]

i figured out how to auth myself againt BB api. I will file initial WIP pull-request today

[iamstarkov]

its happening https://github.com/renovateapp/renovate/pull/1226

[jribeiro]

Awesome @iamstarkov! I've subscribed to the PR. Happy to help testing if needed

[iamstarkov]

its happening.

[iamstarkov]

@jribeiro sure, thats good. we will need QAs on it

[dominique-mueller]

We would also be happy to test this within our organization. Will private BitBucket installations be supported as well?

[iamstarkov]

@dominique-mueller do you mean BitBucket Server?

[iamstarkov]

if it is so, then its our next step to implement support for BB server

[dominique-mueller]

Yep, BitBucket Server it is - can't wait to try it out :)

[binury]

Happy to help with this

[alfaproject]

Happy to help/test on a Bitbucket Server instance (:

[gnujoow]

cant wait for this XD

[azz]

Also happy to help on bitbucket server support. I added bitbucket server support to DangerJS, and that work included adding TS types for a lot of the API responses, which may be beneficial here.

https://github.com/danger/danger-js/blob/master/source/danger.d.ts#L61

[rarkins]

If anyone is interested to help test, the code in #2238 runs now.

Clone the repo, check out the bitbucket branch, run yarn, and then yarn start .....

You will need authentication, see https://github.com/renovatebot/renovate/blob/234b910b2b98ae19cfa3a38c4e623de6889f431f/lib/platform/bitbucket/README.md for details

So it might look like one of these:

BITBUCKET_TOKEN=abc yarn start --log-level=debug --platform=bitbucket thatsme/myrepo
yarn start --platform=bitbucket --username=thatsme --password=something --log-level=debug thatsme/myrepo

It has been tested against BitBucket Cloud for now, when we are happy with it being good enough then it will be merged into master. In theory nearly everything should work with BitBucket Server too and feedback on that compatibility is very welcome. We'll decide based on the complexity whether to fix server compatibility in the same branch or wait until after the merge.

[renovate-bot]

:tada: This issue has been resolved in version 13.52.0 :tada:

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot :package::rocket:

[rarkins]

Bitbucket support has now been merged into master, and ready for beta testing.

For details, see:
https://renovatebot.com/blog/bitbucket-cloud
https://github.com/renovatebot/renovate/blob/master/lib/platform/bitbucket/README.md
https://github.com/renovatebot/renovate/blob/master/docs/self-hosting.md

If you have any config questions or trouble getting started, please post an issue to the dedicated Config Help repo instead: https://github.com/renovatebot/config-help

[mrsegen]

More up to date links regarding BitBucket:
https://github.com/renovatebot/renovate/blob/master/docs/usage/self-hosting.md

Are there more thorough/comprehensive examples out there for BitBucket Cloud?

[viceice]

Please do not post in old issues. Open a new discussion for request help

https://github.com/renovatebot/docker-renovate/blob/master/docs/bitbucket.md