Renovate: Pin and Maintian Circle CI Orb References

[hutson]

What would you like Renovate to be able to do?

To _pin_, and then maintain, Circle CI Orb references in .circleci/config.yml configuration files.

Describe the solution you'd like

My orbital-mechanics package currently references my hutson/library-release-workflows Orb.

It references version 4.1.0, though the latest version of my Orb is 4.1.6.

I would like to see a pull request from Renovate that upgrades the version from 4.1.0 to 4.1.6, such as this example pull request - https://github.com/hyper-expanse/orbital-mechanics/pull/3/files

Describe alternatives you've considered

The goal is to automatically upgrade the Orb version, without resorting to manual intervention. Other than implementing a cron script, I don't know of another way to automate version upgrading of Orb references.

An alternative is to use the volatile label to instruct Circle CI to always use the latest version of the Orb. However, my build would then be non-deterministic.

Additional context

Circle CI does not have an API to fetch version metadata for Orbs.

Support has been requested - https://circleci.ideas.aha.io/ideas/CCI-I-719

However, you can use the Circle CLI (once logged in) to fetch Orb version metadata:

echo '{orb(name:"hutson/library-release-workflows"){name, homeUrl, versions {version, createdAt}}}' | circleci query -

Thanks to Eddie from Circle CI for the solution.

Proof of concept:

'use strict';

const shell = require(`shelljs`);
const semver = require(`semver`);
const yaml = require(`js-yaml`);

shell.config.silent = true;

const circleCIConfig = `
version: 2.1

orbs:
  orb-tools: circleci/orb-tools@volatile
  release-workflows: hutson/[email protected]
`;

const parsedOrbContent = yaml.safeLoad(circleCIConfig);

for (const prop in (parsedOrbContent.orbs || {})) {
  const [orbName, existingVersion] = parsedOrbContent.orbs[prop].split(`@`);

  if (semver.valid(existingVersion) === null) {
    /** Supported labels:
     *
     * circleci/python@volatile
     * circleci/python@2
     * circleci/[email protected]
     * circleci/[email protected]
     * circleci/python@dev:branch
     *
     * Pulled from https://circleci.com/docs/2.0/creating-orbs/#semantic-versioning-in-orbs
     */
    continue;
  }

  const orbVersionMetadata = JSON.parse(shell.exec(`echo '{orb(name:"${orbName}"){name, homeUrl, versions {version, createdAt}}}' | circleci query -`).stdout);

  const latestVersion = orbVersionMetadata.orb.versions.map(versionMetadata => versionMetadata.version).sort(semver.rcompare).shift();

  parsedOrbContent.orbs[prop] = `${orbName}@${latestVersion}`;
}

console.log(yaml.safeDump(parsedOrbContent));

/*
version: 2.1
orbs:
  orb-tools: circleci/orb-tools@volatile
  release-workflows: hutson/[email protected]
*/

[rarkins]

@hutson thanks for submitting the idea. I assume that the CircleCI CLI must just be using an undocumented API to fetch this data?

[rarkins]

Never mind, I located it. Implementation is done, just awaiting testing.

[renovate-bot]

:tada: This issue has been resolved in version 13.152.0 :tada:

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot :package::rocket:

[rarkins]

@hutson I checked and confirmed that Renovate has already updated that reference for you: https://github.com/hyper-expanse/orbital-mechanics/commit/021544e05bc0dc8ead8bf67ca6074306b826d0a5

[hutson]

I assume that the CircleCI CLI must just be using an undocumented API to fetch this data?

Looks like you already found the GraphQL end-point. I was basically going to say, "yeah, they probably are just calling out to an undocumented URL"

I was also hoping to mention that my implementation was lazy, in that I used YAML parsing instead of find-and-replace. I definitely prefer your approach since your approach respects blank lines (which I use liberally in my YAML files for readability).

[rarkins]

I definitely prefer your approach since your approach respects blank lines (which I use liberally in my YAML files for readability).

I learnt pretty quickly that parsing and then stringifying basically never works as you'd want - i.e. it imposes the "opinions" of the stringifier. That's why we go for a more "surgical" approach for json, toml, yaml, etc.

[hutson]

Also, thank you @rarkins for the implementation! It's a sweet new feature to complement my use of Circle CI's new Orbs feature.