Yarn: Undeterministic version sort order in lockfile

[csabapalfi]

Do you want to request a _feature_ or report a _bug_?

Bug

What is the current behavior?

If a transitive dependency version is specified using X (e.g. 3.X.X) in multiple dependencies (and inconsistently, sometimes 'x') then lowercase and uppercase version are sorted randomly in the lockfile. This results in unwanted changes to the yarn.lock file (and git marking it dirty)

If the current behavior is a bug, please provide the steps to reproduce.

git clone https://github.com/csabapalfi/yarn-undeterministic-sort
yarn 
rm -rf node_modules && yarn

Repeat the last step a few times and you should see the yarn.lock file changing even if it doesn't need to.

What is the expected behavior?

Depency version are always sorted the same way.

https://github.com/yarnpkg/yarn/blob/master/src/util/misc.js#L5 should work without lowercasing first but I'm not sure if any other usages of it depend on that.

Please mention your node.js, yarn and operating system version.

• yarn: 0.16.1
• node: v6.7.0, v6.9.0
• OSX: 10.11.6, 10.12

[csabapalfi]

Some hapijs modules are affected by this:
https://github.com/hapijs/heavy/blob/master/package.json#L18
https://github.com/hapijs/h2o2/blob/master/package.json#L18

[kentor]

yep.. seeing stuff like this

-convert-source-map@^1.1.0, convert-source-map@^1.1.1, [email protected], [email protected]:
+convert-source-map@^1.1.0, convert-source-map@^1.1.1, [email protected], [email protected]:
   version "1.3.0"
   resolved "https://registry.yarnpkg.com/convert-source-map/-/convert-source-map-1.3.0.tgz#e9f3e9c6e2728efc2676696a70eb382f73106a67"

[zertosh]

@Daniel15 I've seen this too. I think it's because the sorting is based on the locale https://github.com/yarnpkg/yarn/blob/abb4f7f048a319695aa940c5753469fd606feaeb/src/util/misc.js#L7

[zertosh]

Yikes - @bestander and I are getting different results for this snippet on different machines.

var str = [
  'babel-plugin-transform-es2015-parameters@^6.5.0',
  '[email protected]',
  'babel-plugin-transform-es2015-parameters@^6.7.0',
  'babel-plugin-transform-es2015-parameters@^6.8.0',
  'babel-plugin-transform-es2015-parameters@~6.8.0',
  '[email protected]',
];

function sortAlpha(a, b) {
  // sort alphabetically
  return a.toLowerCase().localeCompare(b.toLowerCase());
}

console.log(str.slice().sort(sortAlpha))

[Daniel15]

Why is it using the locale? Things like this that need to be deterministic
should always use an invariant locale.

Sent from my phone.

On Nov 9, 2016 8:38 AM, "Andres Suarez" [email protected] wrote:

Yikes - @bestander https://github.com/bestander and I are getting
different results for this snippet on different machines.

var str = [
'babel-plugin-transform-es2015-parameters@^6.5.0',
'[email protected]',
'babel-plugin-transform-es2015-parameters@^6.7.0',
'babel-plugin-transform-es2015-parameters@^6.8.0',
'babel-plugin-transform-es2015-parameters@~6.8.0',
'[email protected]',
];
function sortAlpha(a, b) {
// sort alphabetically
return a.toLowerCase().localeCompare(b.toLowerCase());
}
console.log(str.slice().sort(sortAlpha))

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/1308#issuecomment-259460203, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAFnHdQtsYjFRxOi6fYhgtxZJj5bUIfXks5q8fbZgaJpZM4KcT3d
.

[csabapalfi]

Also .toLowerCase() makes sorting versions like 3.x.x and 3.X.X non-deterministic.

[zertosh]

I tried this https://github.com/kosmobot/locale-compare-polyfill/blob/5eab881/src/locale-compare.js#L4 on Node 5.10.0 from nodejs.org and a custom built v5.10.0 (tp2). On the nodejs.org version I get -1, and on the custom build I get 136.

We don't really care what the order is, so long as it's always consistent.

[zertosh]

I'll submit a PR to do this:

export function sortAlpha(a: string, b: string): number {
  // sort alphabetically in a deterministic way
  let aChar;
  let bChar;
  let shortLen = Math.min(a.length, b.length);
  for (let i = 0; i < shortLen; i++) {
    aChar = a.charCodeAt(i);
    bChar = b.charCodeAt(i);
    if (aChar < bChar) {
      return -1;
    } else if (aChar > bChar) {
      return 1;
    }
  }
  if (a.length < b.length) {
    return -1;
  } else if (a.length > b.length) {
    return 1;
  }
  return 0;
}

[kentor]

shouldn't the default sort() (assuming stability) be deterministic?