I've noticed that post-PR-merge CI fails to successfully run the unit tests a few times recently due to an early failure in the unit test suite.
The failing test is from @theia/application-package and is named IsOutdated. It timesout sometimes on Travis, once the 10000ms timer is expired.
This test is run 3 times with different inputs - i've seen 2 or 3 fail on different runs.
For example here 2/3 failed: (https://travis-ci.com/github/eclipse-theia/theia/jobs/366717909#L3896-L3914)
theia/application-package: 2) @theia/[email protected]
@theia/application-package: 4 passing (22s)
@theia/application-package: 2 failing
@theia/application-package: 1) extension-package
@theia/application-package: isOutdated
@theia/application-package: @theia/core@next:
@theia/application-package: Error: Timeout of 10000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/travis/build/eclipse-theia/theia/dev-packages/application-package/lib/extension-package.spec.js)
@theia/application-package: at listOnTimeout (internal/timers.js:549:17)
@theia/application-package: at processTimers (internal/timers.js:492:7)
@theia/application-package: 2) extension-package
@theia/application-package: isOutdated
@theia/application-package: @theia/[email protected]:
@theia/application-package: Error: Timeout of 10000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/travis/build/eclipse-theia/theia/dev-packages/application-package/lib/extension-package.spec.js)
@theia/application-package: at listOnTimeout (internal/timers.js:549:17)
@theia/application-package: at processTimers (internal/timers.js:492:7)
@theia/application-package: Exit with failure status (2): nyc mocha --opts ../../configs/mocha.opts "./lib/**/*.*spec.js"
The failing test checks npm packages versions to determine if a package is outdated vs a proposed versiob. When I run locally, each of the testOutdated waits about 4 seconds for npm to respond - with the timeout of 10s, it runs ok. But on Travis is could be slower, such that 10s timeout is cutting it a bit too close.
Unfortunately it seems left to (bad) luck. I suggest watching CI for newly merged PR for failures, and checking if it's this one. Note that the test suite goes on to do the integration tests ,so one may need to scroll-up quite a bit to reach the beginning of the log for unit tests.
We should wipe those tests; they're unused. ExtensionPackage#isOutdated is called only from the tests; we should remove isOutdated too.
They also depend on npm, maybe there were connectivity issues with the registry again.
Most helpful comment
We should wipe those tests; they're unused.
ExtensionPackage#isOutdatedis called only from the tests; we should removeisOutdatedtoo.