Shields: Badge Request: Azure DevOps Code Coverage

[calebcartwright]

:clipboard: Description

Azure DevOps (formerly known as VSTS) now has publicly accessible APIs that allow for the retrieval of code coverage on builds, so would be great to have a Code Coverage badge for Azure DevOps builds just like the ones that already exist for Jenkins, etc.

I've got a good feel for the APIs involved to make this happen, so I'd love the opportunity to try to add this functionality myself!

• Which service is this badge for e.g: Azure DevOps Pipelines
• What sort of information should this badge show? Code Coverage

:link: Data

• Is there a public API? Yes
• Does the API require an API key? No (for Public Projects), Yes for Private Projects
• https://docs.microsoft.com/en-us/rest/api/vsts/test/code%20coverage/get%20build%20code%20coverage?view=vsts-rest-5.0

[chris48s]

I'd love the opportunity to try to add this functionality myself

Excellent. Sounds like all the data you need is available. We welcome PRs from new contributors. We've got a tutorial on adding new badges that you can follow: https://github.com/badges/shields/blob/master/doc/TUTORIAL.md

We do have a couple of existing badges using the Azure DevOps APIs in https://github.com/badges/shields/blob/master/services/vso/vso.service.js but they use the legacy badge format which we're trying to migrate away from. If you're adding new badges for this service, can you add them using the newer format described in the tutorial. Cheers

[calebcartwright]

Perfect, thanks @chris48s . I'll start working on this once I've had a chance to review the provided docs

[calebcartwright]

Okay I think I've got the gist of this. A few clarification questions:

1. Looks like someone has since renamed the vso services into azure-devops services under https://github.com/badges/shields/tree/master/services/azure-devops and updated them to use the new badge format already, correct?
2. Retrieving the code coverage results for a private project will require an account-specific API token. Based on what I've seen in other services, this would only be supported for a self-hosted instance of shields where the user would add his/her specific token to their private/secret.json file, correct?
3. Based on the tutorial info, the code for this coverage badge should be placed into its own file within the azure-devops directory. I'm assuming this file should be called azure-devops-coverage.service.js given the file names used for other services (i.e. Jenkins), correct?

[paulmelnikow]

1. Looks like someone has since renamed the vso services into azure-devops services under https://github.com/badges/shields/tree/master/services/azure-devops and updated them to use the new badge format already, correct?

That was I and you're correct!

1. Retrieving the code coverage results for a private project will require an account-specific API token. Based on what I've seen in other services, this would only be supported for a self-hosted instance of shields where the user would add his/her specific token to their private/secret.json file, correct?

Yea, that's right. The exception would be if the service offers a "status only" token and there's a way we can validate that's what we've been handed, so we know we're not at risk of exposing any sensitive data.

1. Based on the tutorial info, the code for this coverage badge should be placed into its own file within the azure-devops directory. I'm assuming this file should be called azure-devops-coverage.service.js given the file names used for other services (i.e. Jenkins), correct?

Yep, that's a good place for it!

Sounds like you've got this well in hand, though if you have any other questions, don't hesitate to ask.

[calebcartwright]

Okay I've got the initial implementation working (still need to add examples, tests, etc.). One potential concern I have though is that the Azure DevOps APIs involved with this are still technically in "preview" mode which means there's a chance they could change. In my experience with preview APIs on Azure DevOps they're pretty stable, but Microsoft may tweak the json response and/or query params.

How do you all feel about that? I'm willing to pre-volunteer myself to make updates if the underlying APIs change, but wasn't sure if there were any reservations about adding a badge that relied on service APIs that are still in preview.

[paulmelnikow]

We don't have a hard-and-fast rule about that. I think since it's a widely used product, I'm comfortable using a preview API. It's awesome that you're up for keeping it working as the API develops.

[calebcartwright]

I use the shields service extensively, so i'm happy to give back in any way i can :)

[paulmelnikow]

Well, don't be shy. There's plenty of work to do!

[calebcartwright]

Finished adding tests, will finally get this PR opened within the next day or two

[calebcartwright]

Hmmm So all the PR checks passed, including the tests I'd added (see https://circleci.com/gh/badges/shields/23363) but once I updated my branch with the latest the tests are failing with an error message Route not defined for ${this.name}.

[calebcartwright]

Ah i think I see it, looks like the function url was renamed to route in the base service. Sorry about that, all green checks now!

[paulmelnikow]

Closed via #2327. Nice work!

[DeanPDX]

This is awesome and I'm using it on my project. It's not a huge issue for me (since my particular project is not very public), but, if a build is currently running it shows 0% coverage:

I noticed you are using the build list already to get the latest build ID. I can see two easy fixes:

• Filter build list to only builds where status is completed here.
  OR
• If latest build status is inProgress, throw new NotFound({ prettyMessage: 'build in progress' }).

What do you think? This is a very minor change and I would be willing to submit a pull request if either of these options sounds good. This applies to the Azure test badge as well.

[calebcartwright]

That is an intriguing idea! It sounds worthwhile to me, but I'll defer to the shields members
on this thread @paulmelnikow @chris48s (not sure if there's a mentionable org/team that includes all of you 😄). I don't remember what the behavior is for coverage badges for other services like Jenkins (or frankly if matters whether the coverage result for in-progress builds is consistent across services)

[paulmelnikow]

Why don’t you figure out what makes sense 😉

[calebcartwright]

Go for it @DeanPDX

[ikamensh]

So is the code coverage badge available? How to add it? I am using Azure Pipelines in Azure Devops to build my open source project, I can access the build badge (succeeds / fails) as in the screenshot:

Where do I find the coverage badge?

[calebcartwright]

Hi @ikamensh in the future it would be best to open up a new issue with your question instead.

The coverage badge was added to Shields.io, not to the Azure DevOps platform itself, so you won't find that within the Azure DevOps interface. Try going to https://shields.io/ and browsing the Azure DevOps badges (you can type azure devops in the search field at the top).

It's also the first badge listed at https://shields.io/category/coverage. The badge form is:

https://img.shields.io/azure-devops/coverage/{{org name}}/{{team project name}}/{{pipeline/definition id}}/{{optional branch name}}

For example:
https://img.shields.io/azure-devops/coverage/swellaby/opensource/25

If you run into any issues or have any other questions, please feel free to open a new issue and we'll be happy to help!